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PREFACE 


In  1983  the  Secretary  of  Defense  created  a  new  defense  agency,  the 
Training  and  Performance  Data  Center  (TPDC).  The  Center  is  intended  to 
serve  as  a  repository  for  data  on  a  wide  variety  of  activities  for 
officers  and  enlistees,  actives  and  reserves.  The  data  will  cover  many 
aspects  of  individual  and  unit  training  and  include  information  on 
training  equipment  and  teaching  aids  such  as  simulators.  As  they  become 
available,  data  on  individual  job  performance  and  unit  productivity  will 
also  join  the  TPDC  database. 

TPDC  faces  the  task  of  identifying  the  highest  payoff  data  to 
collect.  In  addressing  this  task,  TPDC  has  sought  guidance  from  the 
services  on  key  training  issues  and  data  availability  and  is  sponsoring 
a  survey  of  their  relevant  databases.  Further,  TPDC  requested  that  RAND 
describe  the  kind  of  data  that  would  be  particularly  useful  in  the  area 
of  initial  skill  training  of  individuals--the  subject  of  this  Note. 

This  research  was  sponsored  by  the  Office  of  the  Assistant 
Secretary  of  Defense  for  Force  Management  and  Personnel.  The  research 
was  conducted  by  the  Defense  Manpower  Research  Center,  part  of  RAND's 
National  Defense  Research  Institute,  a  Federally  Funded  Research  and 
Development  Center  sponsored  by  the  Office  of  the  Secretary  of  Defense. 
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SUMMARY 


In  light  of  its  past  research  in  manpower,  personnel,  and  training. 
The  RAND  Corporation  was  asked  to  suggest  training  and  performance  data 
that  the  Training  and  Performance  Data  Center  (TPDC)  should  collect  in 
an  ideal  training  database.  This  Note  presents  staff  recommendations 
and  guidelines  on  the  kind  of  data  that  should  be  collected  on  initial 
skill  training  (1ST),  discusses  possible  applications  of  the  data,  and 
gives  advice  on  constructing  an  effective  database. 

TPDC  requires  a  database  that  is  comprehensive  and  that  allows  for 
maximum  flexibility  in  the  storing  and  use  of  data  files.  Such  a 
database  would  promote  the  efficiency  and  effectiveness  of  defense 
training  programs.  The  nascent  TPDC  database  could  be  improved  to 
facilitate  policy  analysis  and  reporting,  providing  management  benefits 
for  the  Office  of  the  Secretary  of  Defense  as  well  as  the  services. 

Just  as  the  Defense  Manpower  Data  Center  built  a  database  that  has 
proved  highly  valuable  over  the  last  decade,  so  the  TPDC  could  develop  a 
database  that  would  be  extremely  useful  for  many  purposes. 

TOWARD  A  USEFUL  DATABANK  ON  INITIAL  SKILL  TRAINING 

Some  principle  of  selection  is  needed  for  collecting  data  from 
among  the  myriad  of  initial  skill  training  and  training-related  data. 

The  principle  should  guide  the  collection  effort  in  order  to  compile 
subsets  of  information  adequate  to  support  many  different  topics  and 
analyses.  We  recommend  the  following  principle:  Training  should  be 
considered  an  investment  in  human  capital  meant  to  sustain  or  increase 
military  capability.  As  a  guide  for  selecting  data  for  the  TPDC 
databank,  this  principle  suggests  that  data  should  be  collected  on 
aspects  of  personnel  training  that  relate  to  or  affect  military 
capability.  This  orientation  is  broader  than  the  ordinary  description 
of  training  activities  and  requires  that  trained  personnel  be  followed 
into  the  field,  in  order  to  judge  the  effect  of  their  training  as  it  is 
applied  on  the  job.  A  database  rich  enough  for  analyzing  the 
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relationship  between  training  and  capability  would  also  support  many 
other  studies  and  most  existing  reporting  requirements.  Further,  the 
principle  suggests  that  data  would  be  desired  in  at  least  five 
categories  relating  to  five  types  of  policy  functions. 

•  Selection  of  individuals  into  military  occupations 

•  Individual  performance  of  recruits  in  initial  skill  training 

•  Performance  of  personnel  during  duty  assignment 

•  Productivity  on  the  job 

•  Relationship  of  individual  performance  to  unit  performance 

Each  of  these  categories  includes  many  possible  data  elements.  An 
additional,  and  extensive,  category  of  desirable  information  for  1ST 
data  collection  is  training  costs;  such  a  file  would  support  the  costing 
analyses  associated  with  policy  decisions  made  by  the  training  commands. 

It  is  important  to  know  not  only  what  data  to  collect,  but  on  what 
subjects  to  collect  data.  Some  general  data  should  be  collected  for  all 
subjects,  but  given  the  effort  required  to  sustain  an  extensive, 
detailed  database,  additional  collection  should  be  selective,  at  least 
during  the  initial  years  of  database  development.  Rather  than  collect 
data  for  all  specialties,  it  should  be  obtained  for  only  a  selected 
subset  determined  by  factors  such  as  the  cost  of  recruiting  individuals 
into  a  specialty,  the  cost  of  training,  criticality  and  difficulty  of 
the  skill,  and  the  representativeness  of  the  specialty  within  the  skill 
area.  Training  and  training-related  data  should  be  collected  on  both 
the  active  duty  and  reserve  forces. 

To  enhance  the  advantages  of  a  central  database,  it  should  interact 
with  data  sources  such  as  the  services'  training  databases.  Moreover,  a 
TPDC  database  should  permit  maximum  linkage  among  its  files.  Using  four 
identifiers  would  promote  the  desired  linkage  and  permit  great 
flexibility  in  accessing  the  database:  these  identifiers  are  (1)  the 
individual's  Social  Security  Number  (SSN),  (2)  the  Unit  Identification 
Code  (UIC),  (3)  the  Military  Occupation  Specialty  code  (MOS),  and  (4)  an 
identifier  which  we  propose  be  created--an  1ST  class  number  (ISTN) 
unique  for  each  class  given.  These  identifiers  plus  time  period 
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indicators  would  allow  linkages  among  five  primary  IST-related  files 
that  form  the  core  of  the  database:  files  containing  data  related  to 
the  individual,  the  course,  the  occupation,  the  unit,  and  the  training 
resources . 

POSSIBLE  APPLICATIONS  OF  THE  1ST  DATABASE 

The  1ST  database  is  designed  to  contain  information  on  the  course, 
students,  and  training  outcomes.  Variables  relating  to  equipment, 
training  devices,  instructors,  curriculum  tests,  and  student 
characteristics  can  be  combined  to  analyze  the  effects  of  natural 
variation  in  those  variables  on  test  scores  and,  potentially,  subsequent 
outcomes  on  the  job.  The  data  also  provide  an  extensive  quantitative 
background  for  controlled  experiments  dealing  with  changes  in  training 
policies  and  programs.  Moreover,  the  database  can  support  other 
applications,  some  directly  and  others  closely  related  to  training 
issues.  For  example,  the  1ST  database  could  play  a  significant  role  in 
the  following  five  areas  of  analysis. 

Determining  Training  Loads  for  Budgeting 

A  better  estimate  of  a  training  course's  attrition  rate-used  to 
compile  the  training  load  for  each  1ST  course  and  justify  manpower 
authorizations  for  training— would  be  possible  if  an  1ST  database  were 
available  to  construct  an  attrition  model.  More  comprehensive  data  on 
occupation  selection  criteria,  on  course-related  information,  and  on 
attrition  policy  would  provide  insight  into  the  causes  of  attrition.  A 
better  model  of  training  (course)  attrition  would  also  help  clarify  how 
many  recruits  are  needed  to  meet  a  given  requirement.  A  clearer 
understanding  of  attrition  determinants  would  help  hold  down  recruiting 
and  training  costs . 

Matching  Weapons  System  Acquisition  and  Training  Requirements 

The  greater  military  capability  that  should  accompany  new  weapons 
systems  could  be  reduced  by  the  inordinate  personnel  demands  of  those 
systems,  which  are  often  complicated  to  operate  and  maintain.  By  law, 
manpower  estimates  must  be  made  during  the  early  stages  of  the  weapons 
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system  acquisition  cycle.  The  1ST  database  could  help  estimate  the 
manpower  requirements  for  a  new  system  by  providing  an  occupation 
profile  for  new  items  and  indicating  the  required  change  in  the  number 
of  trained  personnel  by  occupation,  the  skills  possessed,  and  the  impact 
on  training  load.  The  cost  of  alternative  manning  and  training  options 
should  also  be  computed  to  see  whether  the  marginal  costs  of  accessions, 
training,  compensation,  and  benefits  indicate  that  system  requirements 
should  be  changed. 

Managing  Skill  Shortages 

The  TPDC  databank  for  1ST  could  help  TPDC  build  databases  that 
would  support  service  and  OSD  efforts  to  solve  problems  that  are  a 
routine  part  of  the  services'  planning  and  programming  of  training 
resources.  As  the  database  acquired  information  on  1ST  courses,  number 
of  students,  training  equipment,  equipment  shortages,  and  the  like,  a 
record  of  1ST  responsiveness  would  emerge,  documenting  how  skill 
shortages  were  eliminated  in  the  past  and  indicating  how  to  deal  with 
prospective  or  present  shortages.  To  assess  1ST  flexibility  in 
responding  to  shortages,  it  would  also  be  useful  to  track  the  response 
to  anticipated  and  unanticipated  variations  in  the  training  load.  These 
data  could  form  a  valuable  baseline  for  determining  how  greater  training 
availability  might  facilitate  enlistment  in  less  popular  occupations. 

1ST  cost  data  could  aid  in  the  reenlistment  bonus  allocation  process  in 
times  of  skill  shortages.  Finally,  the  database  could  help  plan  for 
shortages  consequent  to  mobilization. 

Selecting  Personnel  into  Military  Occupations 

The  databank  can  help  determine  whether  the  allocation  of  recruits 
across  military  occupations  is  efficient  and  whether  the  level  of 
efficiency  attained  is  sensitive  to  moderate  changes  in  the  allocation. 
Such  changes  may  reflect  changes  in  eligibility  criteria  or  in  recruit 
"quality"  given  eligibility,  i.e.,  predicted  performance  based  on 
individual  characteristics  observable  at  the  time  of  enlistment.  The 
services  and  OSD  have  work  under  way  to  develop  improved  measures  of  job 
performance  in  selected  skills,  and  the  proposed  TPDC  files  could 
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provide  a  useful  foundation  for  the  continuation  of  such  work  on  a 
regular,  rather  than  special  effort,  basis. 

Evaluating  the  Individual's  Return  to  Military  Training 

Finally,  if  information  were  gathered  about  civilian  employment  and 
earnings  for  personnel  leaving  the  service,  the  TPDC  databank  could 
inform  the  military  about  the  value  of  training  to  the  individual.  A 
side  product  of  this  kind  of  analysis  would  be  results  on  whether 
attrition  is  lower  and  reenlistment  higher  for  personnel  with  superior 
measured  performance. 

CONSTRUCTING  AN  EFFECTIVE  TPDC  DATABASE 

In  the  long-term  development  of  the  1ST  database,  one  aspect  of 
content  and  three  aspects  of  structure  must  be  considered  in 
constructing  a  high-quality,  easily  accessible  database.  The  aspect  of 
database  content  is  caution  about  the  use  of  criterion  testing  as  a 
measure  of  performance:  although  practical  in  many  ways,  the  pass/fail 
system  has  disadvantages  for  use  in  statistical  analysis.  It  is  a  poor 
indicator  of  the  degree  of  competence,  as  it  does  not  differentiate 
among  the  quality  of  those  who  pass.  We,  therefore,  recommend  that  data 
on  pass/fail  scores--the  direct  outcome  of  criterion  testing--be 
accompanied  by  data  on  raw  scores  and  perhaps  on  the  number  of  attempts 
and  time  elapsed  until  success  or  failure. 

The  first  aspect  of  structure  is  the  necessity  for  interaction  with 
service  databanks .  TPDC  must  work  with  the  services--the  source  of  most 
training  data--to  devise  common  specification  for  data  elements, 
periodicity,  format,  and  transmission.  Also,  protocols  for  transmission 
need  to  be  devised.  Existing  service  databases  can  provide  a  good  start 
for  building  the  TPDC  database  and  for  developing  the  necessary 
protocols . 

The  second  aspect  of  structure  is  the  essential  requirement  that 
the  data  be  longitudinal  to  allow  for  comparisons  of  cohorts,  evaluation 
of  policy  changes,  studies  of  changes  in  skill  requirements  over  time, 
and  similar  analyses.  Since  the  services  may  not  keep  information  for 
long  periods,  the  TPDC  can  be  the  repository  of  long-term  data.  Hence, 
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the  frequency  of  transmission  of  data  from  the  services  to  TPDC  is  a 
crucial  consideration. 

The  third  aspect  to  be  considered  in  constructing  an  effective 
database  is  that  it  must  be  an  intelligent  database .  It  would  have  an 
automated  codebook;  the  data  would  be  automatically  edited,  according  to 
programmed  instructions  and  documented  criteria;  and  the  summary 
statistics  would  be  readily  available  to  browsers.  Information  across 
files  would  be  combined  automatically.  The  database  system  should  keep 
track  of  file  locations  and  be  able  to  write  out  requested  data  in 
specified  formats  to  data  files  useful  for  statistical  analyses. 

CONCLUSION 

The  TPDC  database  on  1ST  should  contain  a  series  of  connectable 
files  that  can  be  linked  in  different  combinations  and  serve  different 
kinds  of  research  and  analysis.  A  central  database  of  the  kind 
described  here  would  have  many  large  benefits  both  for  OSD  and  the 
individual  services. 

Compiling  this  kind  of  database  is  a  vast  effort  that  is  likely  to 
proceed  incrementally  at  best  over  many  years.  We  emphasize  that  the 
successful  creation  of  an  efficient,  flexible,  useful  database  will 
require  the  cooperation  of  all  the  services,  as  well  as  of  OSD.  Their 
cooperation  will  depend  on  two  conditions:  (1)  evidence  that  useful 
studies  can  be  done  from  the  kind  of  data  in  an  1ST  database,  and  (2) 
assurances  that  the  content  and  consequences  of  these  studies  will  not 
threaten  the  services'  mission  to  increase  their  capability.  The 
Defense  Manpower  Data  Center  illustrates  the  value  of  such  cooperation; 
its  data  files  have  provided  a  detailed  record  of  personnel  trends  and 
the  effects  of  policy  ind  economic  changes  on  those  trends,  and  both  the 
OSD  and  the  services  have  used  the  files  for  many  valuable  studies. 

The  architecture  we  propose  for  the  TPDC  databank  would  accommodate 
various  kinds  of  information  from,  and  useful  to,  all  the  services.  The 
services  should  find  many  fruitful  uses  for  this  kind  of  database  as  it 
grows  and  evolves.  Furthermore,  it  should  support  experimentation  in 
different  kinds  of  research  and  analysis  and  yield  heretofore  unexpected 
findings  of  benefit  to  military  managers  and  planners. 
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I.  INTRODUCTION 


The  Training  and  Performance  Data  Center  (TPDC)  has  asked  RAND  for 
advice  on  developing  an  "ideal"  database  on  initial  skill  training.  In 
response,  this  Note  documents  recommendations  on  the  kind  of  data  that 
ought  to  be  collected,  discusses  diverse  applications  of  the  data,  and 
suggests  requirements  for  constructing  an  effective  database.  RAND's 
advice  and  recommendations  were  communicated  to  TPDC  in  1985,  and  the 
subsequent  interaction  between  RAND  and  TPDC  indicates  that  TPDC's  data 
collection  strategy  has  progressed  along  lines  consistent  with  RAND's 
suggested  approach.  We  suggest  that  a  TPDC  database  on  initial  skill 
training  should  contain  a  series  of  connectable  files  that  can  be  linked 
in  different  combinations  and  be  suitable  for  different  kinds  of 
research  and  analysis.  A  central  databank  of  the  kind  recommended  here 
would  have  benefits  both  for  the  Office  of  the  Secretary  of  Defense 
(OSD)  and  the  individual  services. 

A  primary  purpose  of  TPDC  is  to  be  the  DoD  focal  point  for  defense 
training  data,  and  to  become  a  main  repository  for  that  data.  It  was 
set  up  in  1984  to  collect  and  maintain  integrated  training-related 
information  and  make  it  available  to  the  DoD  training  community.  Data 
selected  for  the  TPDC  database  must  be  current  and  useful,  and  the 
methods  of  collecting  and  storing  the  data  must  allow  for  maximum 
interaction  among  data  files.  In  short,  if  it  is  to  help,  the  database 
must  be  comprehensive  and  flexible.1 

In  general  purpose  and  usefulness,  TPDC  is  somewhat  analogous  to 
the  Defense  Manpower  Data  Center  (DMDC).  In  fact,  the  creation  of  TPDC 
has  certain  parallels  to  the  establishment  of  DMDC  a  decade  earlier. 
Demand  for  the  DMDC  grew  from  the  earlier  national  debate  over  the 
draft,  as  policymakers  and  analysts  were  driven  to  recognize  the  value 
of  a  central  databank  for  defense  manpower  data.  Tfte  lack  of  such  an 

*G.  Thomas  Sicilia,  "Centralizing  the  Management  of  Defense 
Training  Data,"  Defense  Management  Journal,  Vol.  20,  No.  4,  1984,  pp. 
9-11. 
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archive  delayed  analysis  and  limited  the  range  of  issues  that  could  be 
investigated.  As  a  consequence,  the  Gates  Commission  recommended 
establishing  an  agency  that  would  draw  manpower  data  from  all  services, 
and  soon  thereafter  DMDC  was  established  by  the  Secretary  of  Defense. 
DMDC  files  have  improved  continuously  over  time  in  coverage  and  quality; 
they  provide  a  detailed  record  of  personnel  trends  and,  implicitly,  the 
effects  of  policy  and  economic  changes  on  those  trends.  The  large 
number  of  studies  based  on  DMDC  data  attest  to  their  high  value, 
although  the  specific  studies  were  probably  not  foreseen  by  the  framers 
of  the  DMDC  charter. 

The  limited  training  and  performance  data  that  exist  now  are  for 
the  most  part  kept  separately  by  each  service,  as  was  the  case  with 
personnel  data  before  DMDC.  Protocols  to  facilitate  the  sharing  of 
training  information  have  not  been  written.  Except  for  information 
required  in  the  annual  budget  process ,  the  services  have  so  far  had 
little  impetus  to  forward  data  to  OSD,  exchange  it  among  themselves,  or 
supply  it  to  others.  But  now,  with  increasing  interest  in  topics 
concerning  force  structure  and  capability-such  as  Navy  and  Air  Force 
growth.  Army  reserve  growth,  weapons  system  modernization,  and  the  role 
of  manpower  and  training  in  the  weapons  system  acquisition  process-- 
the  circumstances  have  changed.  Defense  planners  have  greater  incentive 
to  learn  about  training  and  its  relationship  to  force  effectiveness 
because  more  trained  and  experienced  personnel  will  be  needed,  even 
though  defense  budgets  are  not  expected  to  grow  as  much  in  real  terms 
over  the  next  five  years  as  they  have  in  the  past  five.  This 
assessment,  first  put  forward  by  the  1982  Defense  Science  Board 
recommendation  supporting  the  creation  of  TPDC,  seems  at  least  as  valid 
today  as  then. 

TPDC  provided  two  guidelines  in  its  request  for  RAND  to  help 
organize  its  data  holdings:2 


2A  separate  TPDC-sponsored  project  has  reviewed  the  usefulness  of 
existing  service-maintained  databases  in  constructing  a  training 
database;  see  Hammon  and  Horowitz  (1987). 
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•  To  focus  on  initial  skill  training  and 

•  To  consider  data  appropriate  for  research  and  policy  analysis, 
without  limiting  the  discussion  to  currently  available  data. 

To  this  end,  Sec.  II  defines  initial  skill  training  and  lists  a  general 
set  of  data  files  worthwhile  establishing  for  an  1ST  database.  Section 
III  discusses  a  number  of  policy-related  applications  that  a  training 
and  performance  database  might  be  expected  to  support,  and  the 
concluding  section  discusses  factors  leading  to  the  construction  of  an 
effective  TPDC  initial  skill  training  database. 
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II.  TOWARD  A  USEFUL  INITIAL  SKILL  TRAINING  DATABANK 


The  TPDC  is  interested  in  data  on  initial  skill  training  (1ST)  in 
the  services  that  might  be  useful  for  research  and  policy  analysis.  We 
begin  this  section  by  defining  initial  skill  training.  We  then  discuss 
a  principle  that  will  help  guide  the  kind  of  data  collection  effort 
called  for  and  help  define  the  data  requirements.  After  discussing 
training  costs,  we  consider  selective  data  collection  and  discuss  how 
the  data  files  can  be  linked. 

INITIAL  SKILL  TRAINING 

All  services  offer  1ST.  As  the  most  common  form  of  specialized 
skill  training  for  enlisted  personnel,  it  involves  instruction  in  job- 
related  technical  specialties.  It  is  usually  given  after  recruit 
(basic)  training  and  before  the  first  duty  assignment.  Each  service 
offers  1ST  in  approximately  300  different  specialities,  and  the  courses 
serve  actives  and  reservists.  For  the  most  part,  a  service  teaches  its 
own  courses,  but  interservice  courses  are  not  uncommon;  for  instance, 
the  Marine  Corps  and  Air  Force  subscribe  to  the  Army  course  in  "Tracked 
vehicle  repair,"  and  both  the  Navy  and  Air  Force  offer  cryptologic 
courses  to  the  other  services.  In  a  few  cases,  such  as  infantryman,  1ST 
has  been  combined  with  recruit  training  into  an  integrated  course 
referred  to  as  "one-station  unit  training"  (OSUT) .  The  duration  of  1ST 
varies  with  the  specialty,  ranging  from  roughly  a  month  to  a  year.  The 
Army  and  Navy  have  specialized  skill  training  facilities  at  over  30 
bases  each,  the  Marine  Corps  at  seven  bases,  and  the  Air  Force  it  nine 
bases.  The  bulk  of  these  facilities  offer  1ST.  In  FY  1987  the  services 
expect  to  supply  1ST  to  454,000  enlisted  personnel  in  active,  guard,  and 
reserve  forces,  producing  420,000  graduates--a  completion  rate  of  92.5 
percent . 1 

lFrom  Military  Manpower  Training  Report,  FY  1987 ,  Volume  IV:  Force 
Readiness  Report,  p.  V-4.  The  Report  is  an  excellent  source  of 
information  about  the  scope  of  individual  and  unit  training  activities. 
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PRINCIPLE  FOR  GUIDING  DATA  COLLECTION 

There  are  many  kinds  of  training  data,  and  data  can  be  used  for 
different  purposes.  Data  on  1ST,  for  example,  can  help  inform  policy- 
related  questions  ranging  from  descriptive  to  analytical.  Should 
Congress  and  the  services  want  to  know  how  many  soldiers  are  being 
trained  where  and  by  whom,  information  is  needed  on  students, 
facilities,  and  instructors.  There  are  questions  about  determining 
training  budgets  and  personnel  flows  (e.g.,  accession  requirements, 
skill  shortages  or  excesses),  and  beyond  those  are  topics  concerning 
course  design,  the  balance  between  schoolhouse  and  on-the-job  training, 
and  training  effectiveness.  A  training  database  should  be  capable  of 
supporting  such  a  variety  of  topics.  But  among  the  myriad  of  training 
and  training-related  data  that  exists,  how  does  one  determine  which  data 
are  needed  or  most  useful? 

Some  principle  must  guide  the  data  collection  effort  in  order  to 
select  a  subset  of  information  adequate  to  support  many  topics.  We 
suggest  the  following  principle:  Training  should  be  considered  an 
investment  in  human  capital  meant  to  sustain  or  increase  military 
capability.  As  a  guide,  this  principle  directs  that  data  should  be 
collected  on  aspects  of  training  that  relate  to  or  affect  military 
capability.  In  particular,  the  principle  extends  the  scope  of  training 
data  collection  beyond  training  schools  and  into  the  field.  It  thereby 
provides  a  more  complete  quantitative  vision  of  training  and  its 
consequences  than  would  otherwise  be  available,  or,  for  that  matter, 
appears  to  be  available  in  today's  training  data  systems.  We  use 
"military  capability"  in  a  broad  sense,  including  tactical  and  strategic 
activities  involved  in  force  employment  and  deterrence,  and  the  many 
support  and  service  functions  needed  to  sustain  them. 

The  principle  stipulates  broad  data  requirements.  In  all 
likelihood,  a  database  rich  enough  to  explore  the  relationship  between 
training  and  capability  would  also  support  other  studies  and  most 
existing  reporting  requirements.  The  principle  also  governs  long-term 
data  collection:  Major  portions  of  desired  data,  particularly  data  on 
job  performance  and  productivity,  are  not  readily  available  today,  but 
it  is  important  to  know  what  those  future  requirements  are. 
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Obtaining  the  required  data  will  be  burdensome,  but  linking  the 
data  on  training  and  capability  should  have  a  substantial  payoff:  Such 
data  can  yield  insight  into  the  effectiveness  of  alternative  defense 
resource  allocations,  as  witnessed  in  practice,  and  can  help  support 
experiments  for  the  same  purpose.  Results  from  analyses  based  on  the 
data  should  be  useful  for  existing  weapons  systems  and,  by 
extrapolation,  for  new  ones  in  the  process  of  acquisition. 

The  principle  further  suggests  that  data  would  be  desired  in  at 
least  five  categories: 

•  Selection  of  individuals  into  military  occupations 

•  Individual  performance  of  recruits  in  initial  skill  training 

•  Performance  of  personnel  during  duty  assignment 

•  Productivity  on  the  job 

•  Relationship  of  individual  performance  to  unit  performance 

Each  of  these  categories  includes  many  possible  data  elements,  and 
decisions  will  have  to  be  made  about  which  elements  to  collect.  For 
instance,  across  the  services  there  are  over  a  thousand  distinct 
occupations,  and  every  occupation  can  be  described  in  terms  of  its 
required  duties  and  eligibility  criteria,  which  may  change  over  time. 
Further,  as  operation  and  maintenance  patterns  change  or  new  weapons 
systems  are  phased  in,  occupations  may  be  created  or  eliminated.  The 
duty  content  of  an  occupation  can  thus  change  gradually  or  abruptly  over 
time,  as  can  the  qualifications  of  entering  personnel.  Similarly,  other 
data  elements  include  the  many  factors  bearing  on  1ST  performance.  It 
would  be  useful  to  have  data  collected  from  each  class  on  individual 
student  characteristics,  instructor  characteristics,  course  content, 
course  and  training  devices,  tests  and  standards,  individual  student 
performance  including  test  scores  and  class  rank,  and  overall  class 
performance.2  Training  devices  include  examples  of  field  equipment 

2Detailed  class  information  also  helps  ameliorate  the  problem  of 
variation  in  course  instruction  and  tests  among  training  schools.  A 
study  of  11  Array  specialty  training  courses  conducted  by  the  Army 
Research  Institute  (Oxford-Carpenter  et  al.,  1984)  found  that  in  some 
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(radios,  rifles,  armor,  antennas,  tools,  parts,  etc.)  and  simulators  and 
other  teaching  aids  (movies,  interactive  video  disks).  Many  tests  are 
given  during  1ST,  some  hands-on  and  some  written,  and  the  content  of 
these  tests  will  change  from  year  to  year  as  changes  in  weaponry, 
tactics,  and  mission  occur.1  Documentation  of  such  changes  is  needed, 
for  example,  when  course  planners  want  to  understand  why  the  concordance 
between  1ST  performance  and  job  performance  might  have  become  stronger 
or  weaker. 

Specifying  data  for  job  performance  and  productivity  is  equally 
demanding.  Job  performance  underlies  productivity  in  the  sense  that 
performance  indicates  proficiency  in  the  skill  for  which  one  has  been 
trained.  A  soldier's  job  performance  is  measured  through  his  capability 
to  accomplish  a  set  of  specialty-specific  tasks;  thus,  it  is  first 
necessary  to  define  the  tasks  and  develop  test  instruments.  To  analyze 
job  performance  as  a  function  of  1ST  performance,  data  on  other, 
possibly  correlated,  factors  such  as  job  experience  and  on-the-job 
training  must  also  be  obtained.  These  factors  raise  questions  of  item 
definition  in  the  course  of  database  development --can  experience  be 
measured  simply  as  time  on  duty  assignment?  How  should  on-the-job 
training  be  measured? 

In  contrast  to  job  performance,  which  is  individual -speci f ic ,  the 
notion  of  productivity  extends  to  the  unit  and  the  efficiency  of  its 
configuration  to  accomplish  its  mission.  A  soldier's  competence  to 
perform  his  required  tasks  is  one  element  of  this;  another  is  how  unit 
personnel  compensate  for  existing  deficits  in  equipment,  experience,  or 
training.  The  ability  to  compensate  probably  involves  familiarity  with 
the  equipment,  shared  experience,  an  ability  to  assign  correct 
priorities  to  a  workload,  and  an  ability  to  make  allocative  decisions 

instances  two  courses  covering  the  same  job  were  taught  differently  at 
different  training  installations,  although  the  same  final  test  was 
administered.  The  study  found  many  differences  across  schools  and 
courses  in  the  way  course  materials  and  test  materials  were  handled. 

’From  the  viewpoint  of  analyzing  the  effectiveness  of  1ST,  we  at 
RAND  are  concerned  about  the  growing  use  of  criterion  testing  in  which 
the  test  outcome  is  recorded  merely  as  pass  or  fail.  For  further 
discussion  of  criterion  testing  and  the  problem  it  presents,  see  Sec. 

IV. 
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rapidly  and  creatively.  Thus,  measures  that  capture  the  ability  of  a 
whole  unit  are  necessary,  in  addition  to  measures  of  individual  job 
performance  within  the  unit.  To  learn  how  mission  capability  changes  in 
response  to  changes  in  unit  manning,  skill  level  and  mix,  equipment,  and 
facilities,  data  on  these  variables  must  be  recorded  along  with  measures 
of  unit  capability.  A  unit's  mission  capability  may  also  be  affected  by 
the  level  of  on-the-job  training  (OJT)  conducted  within  the  unit  and 
whether  it  has  the  resources  available  to  provide  adequate  OJT.“  Thus, 
unit-level  data  might  include  information  on  the  unit's  monthly  training 
load  and  training  resource  availability. 

What  types  of  data  might  be  useful  in  measuring  productivity  for 
the  unit?  Results  from  field  exercises  may  provide  information  on  the 
ability  of  the  unit  to  meet  its  mission.  The  maintenance  management 
data  systems  within  each  of  the  services  might  provide  productivity 
information  for  units  involved  in  maintenance.  Table  1  lists  the 
maintenance  management  systems  that  provide  the  information  needed  to 

Table  1 

MAINTENANCE  MANAGEMENT  SYSTEMS 


Service 

System 

Level  of  Identification 

Army 

The  Army  Maintenance 
Management  System 
(TAMMS) 

Unit 

Air  Force 

Maintenance  Data 
Collection  System 
(MDC) 

Work 

center 

Navy  and 

Marine  Corps 

Maintenance  and  Material 
Management  System  (3-M): 
one  for  aviation,  one 
for  ships 

Work 

center 

‘Reuter  et  al.  (1980)  found  a  negative  relationship  between  the 
amount  of  training  conducted  by  aircraft  maintenance,  transportation, 
and  supply  units  in  the  Air  Force  and  the  units'  mission  performance. 
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manage  the  maintenance  of  weapons  and  support  equipment  such  as  the 
availability  of  spare  parts  and  the  types  of  equipment  failures.  The 
systems  provide  information  on  the  conduct  of  all  maintenance  tasks-- 
what  equipment  was  maintained,  why  maintenance  was  required,  what  was 
done,  how  long  it  took,  and  who  did  it.5  Maintenance  management  file 
data  are  useful  at  the  unit  level,  but  because  they  were  not  designed  to 
evaluate  the  performance  of  maintenance  technicians,  they  may  not 
contain  the  name  of  the  individual  who  performed  the  repair,  and  thus 
may  be  of  limited  use  for  individual  productivity.'  We  recognize  that 
these  systems  are  not  perfect,  but  they  do  represent  a  promising  start. 
Improvements  in  data  validation  and  accuracy  in  recording  how  time  was 
spent  by  maintenance  task  would  contribute  to  the  reliability  of  the 
data . 


Aggregate  level  studies  using  data  from  maintenance  management 
systems  have  shown  that  the  information  contained  within  those  systems 
can  provide  measures  that  reflect  productivity  such  as  readiness  of 
equipment.  A  study  by  Orlansky  and  String  (1981)  suggests  that 
inadequate  performance  by  maintenance  technicians  is  a  factor  that 
contributes  to  the  "not  ready"  status  of  military  equipment.  Horowitz 
and  Sherman  (1980)  used  ship  maintenance  and  ship  personnel  records  to 
examine  the  relationship  between  productivity  (as  measured  by  ship 
downtime  for  a  given  subsystem,  e.g.,  boilers)  and  personnel 
characteristics,  controlling  for  ship  characteristics  which  can  affect 
the  level  of  downtime.  Their  results  suggest  that  more  experienced 
crews  and  crews  with  more  formal  school  training  had  less  ship  downtime. 

'The  potential  usefulness  of  maintenance  data  in  measuring 
productivity  has  led  to  an  interest  in  modifying  such  data  systems  to 
include  technician  identifiers.  These  identifiers  would  permit  the 
linking  of  maintenance  and  training  data  systems.  McConnell  and  Jones 
(1985)  looked  into  the  feasibility  of  combining  maintenance  management 
data  systems  and  training  data  systems  within  the  services.  They 
concluded  that,  generally,  with  the  addition  of  information  to  identify 
individuals  performing  maintenance,  these  systems  could  be  linked  and 
thus  provide  information  necessary  to  carry  out  analysis  of  maintenance- 
related  training.  (However,  it  should  be  noted  that  problems  arise  when 
the  work  on  a  specific  piece  of  equipment  involves  more  than  one  person 
or  is  performed  over  more  than  one  shift.)  In  response  to  this 
interest,  the  Army  Research  Institute  developed  the  Army  Maintenance 
Performance  System  (MPS)  to  record  the  work  experience  (time  on  each 
technical  task  in  the  maintenance  battalion)  and  training  (courses  and 
qualification  tests)  of  each  maintenance  technician.  This  database  may 
provide  useful  information  for  the  TPDC  training  database. 
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The  guiding  principle  of  training  as  investment  in  human  capital 
suggests  an  additional  data  category  not  mentioned  above.  That  category 
deals  with  the  value  of  the  training  to  the  individual  and,  among  other 
things,  requires  data  on  his  post-service  employment  and  earnings.  Such 
data  would  improve  the  ability  to  analyze  the  value  of  military  training 
in  post-service  employment  and  presumably  bear  on  the  individual's 
willingness  to  enter  the  service  in  the  first  place.  It  would  also  help 
depict  the  role  of  the  military  as  a  provider  of  trained  personnel  to 
the  civilian  sector. 

TRAINING  RESOURCES 

Training  resources  comprise  an  additional  category  for  1ST  data 
collection  from  which  training  cost  estimates  can  be  made.  As  with  job 
performance  and  unit  productivity  measurement,  careful  thought  will  be 
needed  to  determine  what  resource  information  ought  to  be  obtained.  The 
appropriateness  of  such  data  depends  on  the  problem  at  hand;  for 
purposes  of  policy  analysis,  for  instance,  it  will  probably  not  suffice 
to  collect  only  the  operations  and  maintenance  resources  of  formal 
training.  These  may  or  may  not  address  the  full  resource  consequences 
of  the  policy  change  and  in  any  case  might  not  represent  the  opportunity- 
cost  of  the  change.  Elaborating,  there  are  many  distinctions  to  keep  in 
mind  when  specifying  the  requirements  for  training  resource  data:  fixed 
versus  variable  resources;  budgetary  versus  economic  for  opportunity) 
cost  of  the  resources;  the  resources  used  in  training  versus  the  total 
resources  used  to  produce  trained  manpower;  and  the  resources  for  formal 
training  versus  on-the-job  training.  Fixed  resources  become  important 
when  new  training  facilities  are  contemplated.  They  may  involve  land 
acquisition,  architectural  planning,  construction,  and  major  equipment 
purchases,  for  example.  Since  these  expenditures  will  vary  across  sites 
and  over  time,  it  seems  unrealistic  to  propose  that  they  be  maintained 
in  a  routinely  updated  file,  as  opposed  to  being  collected  as  needed  on 
a  case  by  case  basis.  Still,  once  collected,  they  would  afford  valuable 
documentation  of  the  alternatives  evaluated  and  the  decisions  made. 
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The  distinction  between  budgetary  and  economic  costs  poses  a 
dilemma.  Economic  theories  of  decisionmaking,  which  underlie  much 
policy  analysis,  emphasize  that  the  relevant  costs  oi  a  decision  are 
opportunity  costs,  i.e.,  the  costs  of  the  best  forgone  (alternative)  use 
of  the  resources  involved.  But  economic  cost  data  are  typically  not 
readily  available,  and  indeed  considerable  time  and  effort  may  be 
required  to  obtain  them.  Budgetary  costs,  in  contrast,  often  are 
available,  yet  unfortunately  they  may  bear  little  relation  to  the 
economic  costs  of  a  policy  action.  For  example,  if  ten  trainers  are 
added  to  a  training  brigade,  the  budgetary  cost  to  the  training 
establishment  equals  their  military  compensation.  (If  endstrength  is 
unchanged  and  the  personnel  were  merely  transferred  from  another 
activity,  the  budgetary  cost  to  the  service  is  unaffected  because  the 
total  number  of  people  on  the  payroll  is  the  same.)  But  the  economic 
cost  of  the  ten  trainers  is  the  value  of  their  best  alternative  use; 
this  shadow  price  could  be  higher  or  lower  than  the  budgetary  cost.  The 
trainers  conceivably  might  have  been  diverted  from  a  crucial  mission,  or 
they  might  have  been  underutilized  in  their  previous  duty. 

Budgetary  costs  can  be  obtained  from  the  various  accounting  systems 
that  keep  track  of  expenditures  under  current  year  budgets  and  aid  in 
justifying  outyear  budget  requests.  At  times  budget  data  are  assumed 
close  enough  to  economic  costs  to  rationalize  their  use  in  a  policy  cost 
analysis.  Whereas  their  appropriateness  is  an  issue  for  analysts  and 
policymakers  to  decide  when  evaluating  specific  policy  alternatives,  it 
seems  advisable  for  TPDC  to  develop  files  of  training  resources  and 
related  files  of  their  budgetary  cost.  The  files  might  itemize 
personnel,  facilities,  and  equipment,  and  do  so  by  course.  Subsequent 
effort  might  focus  on  estimating  resources  applied  to  on-the-job 
training . 

The  term  "training  cost"  is  frequently  used  in  reference  to  the 
cost  of  trained  manpower,  but  the  two  kinds  of  cost  may  be  distinct. 

The  cost  of  trained  manpower  includes  recruiting,  formal  and  on-the- 
job  training,  personnel  compensation,  and  change  of  station.  It  is  a 
lifecycle  cost  concept,  whereas  training  cost  typically  refers  to  a 
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training  course.  For  instance,  consider  a  directive  requiring  five 
percent  more  trainees  from  a  course,  to  be  achieved  through  lower 
attrition.  The  training  cost  of  this  directive  would  include  the 
incremental  1ST  resources  required  for  the  additional  graduates.  If  the 
incremental  cost  per  graduate  of  these  resources  equalled  the  average 
cost  per  graduate,  and  if  post-course  attrition  remained  constant,  the 
incremental  cost  of  trained  manpower  would  not  change.  However,  if 
marginal  cost  exceeded  average  cost  or  if  post-course  attrition  rose, 
the  incremental  cost  of  trained  manpower  would  increase.  Thus,  training 
cost  can  refer  to  training  resources  and  how  they  vary  as  the  scale  or 
content  of  training  changes  (e.g.,  number  of  trainees,  instructors,  or 
amount  and  kind  of  training  equipment),  while  the  cost  of  trained 
manpower  is  a  lifecycle  concept  about  the  replacement  cost  of  trained 
personnel  with  given  years  of  experience. 

The  distinction  between  formal  and  on-the-job  training  is  well 
known  but  difficult  to  implement.  Tt  is  generally  thought  easier  to 
develop  resource  estimates  for  formal  training  because  training  is  the 
exclusive  activity,  whereas  on-the-job  training  may  be  a  joint  product 
with  the  unit's  regular  activities.  As  a  result,  special  assumptions 
may  be  required  to  estimate  the  incremental  or  total  resources  related 
to  on-the-job  training--again,  this  will  require  careful  consideration. 

SELECTIVE  DATA  COLLECTION 

The  level  of  detail  suggested  for  the  1ST  database  precludes 
collection  for  all  individuals  and  specialties.  Thus,  thought  must  be 
given  not  only  to  what  data  to  collect  but  on  what  subjects  to  collect 
data.  While  some  general  training  data  may  be  collected  for  all,  more 
detailed  informat ion--especial ly  that  related  to  post-training 
performance  and  productivity--should  be  obtained  only  for  a  selected 
subset  of  specialties.  That  selection  could  be  based  on  factors  such  as 
the  cost  of  recruiting  individuals  into  the  specialty  f including 
enlistment  incentives),  the  cost  of  training  (including  direct  costs 
plus  student’s  time  and  costs  imposed  by  attrition),  criticality  and 
difficulty  of  the  skill,  and  the  representat iveness  of  the  specialty 
relative  to  other  specialties  in  the  skill  area.  Over  time,  as  the 
database  evolved,  it  might  be  desirable  to  broaden  the  selection. 
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Training  and  training-related  data  should  be  collected  on  both  the 
active  duty  and  the  reserve  forces.  Reserve  training  has  become  a 
prominent  issue  as  the  reserves'  role  in  the  total  force  has  grown.  At 
present,  databases  are  not  adequate  to  analyze  the  effectiveness  of 
reserve  training  and  the  readiness  of  the  reserve  forces.  However,  much 
of  the  data  suggested  for  the  1ST  database  could  be  collected  for  the 
reserves  as  well.  Reservists  attend  the  same  classes  as  their  active 
counterparts  or  may  take  correspondence  courses  for  which  course 
description  data  could  be  obtained.  On-the-job  training  is  a  major 
component  of  reserve  training7  and  job  performance  measures  developed 
for  the  actives  would  apply  as  well.  Reserve  units  have  assigned 
equipment  and  perform  in  periodic  exercises  which  can  provide 
information  on  unit  performance  and  mission  capability.  In  addition, 
personnel  records  are  maintained  for  reservists.  Although  there  are 
some  questions  about  data  quality,  they  would  be  resolved  through 
increased  demand  for  that  data  by  the  training  community,  plus  more 
organized  and  standardized  collection. 

The  above  information  could  be  collected  for  a  selected  subset  of 
reserve  specialties  chosen  by  guidelines  similar  to  those  for  the  active 
forces.  An  overlap  in  specialties  would  be  desirable  to  compare 
training  and  readiness  between  the  actives  and  reserves. 

LINKING  THE  DATA 

The  advantages  of  a  central  databank  lie  in  quick  accessibility  to 
data,  uniformity  of  data  records  across  services  and  skills  and  over 
time,  and  the  concentration  of  individuals  knowledgeable  about  the  data. 
The  database  should  have  connections  to  data  sources,  such  as  the 
services'  training  commands  and  training  bases,  to  handle  questions 
about  data  quality  and  content  and  to  suggest  improvements.  Moreover, 
the  value  of  the  TPDC  database  would  be  greatly  ennanced  by  a  system 
permitting  maximum  linkage  among  the  files. 

7This  refers  to  the  Selected  Reserves;  the  Individual  Ready  Reserve 
has  little  required  refresher  training. 
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Four  identifiers  may  suffice  to  promote  the  desired  linkage  among 
files.  The  first  three  already  exist--the  individual's  Social  Security 
Number  (SSN),  the  Unit  Identification  Code  (UIC),  and  the  Military 
Occupation  Specialty  code  (MOS) . 

We  propose  creating  a  fourth  identifier:  a  unique  number  for  each 
1ST  class  given--an  "ISTN."  The  ISTN  could  encode  information  about  the 
service  offering  the  course,  its  location,  and  date,  and  would  be 
attached  to  the  student's  personnel  record,  along  with  UIC  and  the  MOS. 

Because  unit  and  MOS  characteristics  can  change  over  time,  a  time 
indicator  should  be  included  as  an  additional  identifier.  These 
characteristics  could  be  collected  at  set  intervals,  such  as  every  year, 
for  example,  or  they  can  be  collected  whenever  there  is  a  significant 
change  related  to  policy  or  personnel.  Thus,  the  unit-  and  MOS-related 
files  would  have  multiple  records  per  unit  or  MOS,  each  representing  a 
specific  time  period  or  interval.  The  time  variable  might  simply  be  a 
year  indicator  (or  six  month  or  quarterly  indicator,  depending  on  the 
data)  or  it  might  consist  of  a  begin  date  and  an  end  date  for  the  period 
over  which  the  data  are  relevant.  Personnel  records  should  contain 
dates  as  individuals  move  in  and  out  of  units  and  MOS's,  so  that  unit- 
and  MOS-related  information  can  be  merged  in  from  the  appropriate  time 
period.  The  ISTN  already  would  have  a  time  factor  for  course-related 
data  incorporated  into  its  code. 

The  SSN,  ISTN,  MOS,  and  UIC  plus  time  indicators  would  permit  great 
flexibility  in  accessing  the  database.  For  instance,  matching  personnel 
records  by  ISTN  would  result  in  a  roster  of  students  in  the  class, 
whereas  matching  the  ISTNs  on  an  individual's  record  with  course-related 
files  leads  to  data  describing  course  objectives,  skills  taught, 
teaching  aids,  equipment,  length,  location,  time,  instructors 
(themselves  denoted  by  their  SSNs),  tests  and  grading  standards,  and 
attrition  rate.  Matching  class  graduates'  UICs  would  show  the  units 
they  were  assigned  to  and,  implicitly,  whether  these  duty  assignments 
conformed  to  the  training.  In  addition,  DoD  could  house  files  on 
individual  performance  in  1ST  and  on  the  job,  and  these  performance 
files  could  be  linked  via  SSN  to  personnel  records,  class  information, 
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and  unit  information.  The  MOS,  for  example,  allows  linkage  between  "end 
item"  files  containing  MOS  profiles  for  every  end  item  in  the  defense 
inventory  (e.g.,  vehicles,  weapons  systems)  and  personnel,  unit,  and 
training  course  records. 

Table  2  presents  a  list  of  IST-related  files  which  would  form  the 
core  of  the  1ST  database.  Examples  of  the  types  of  data  that  would  be 
included  in  each  file  are  also  listed  to  suggest  the  potential  contents 
of  the  files.  These  files,  with  the  exception  of  personnel  records, 
could  be  kept  at  TPDC*  pending  service  and  OSD  agreement  and 
coordination. 

The  files  can  support  reports  and  analyses  on  the  scope  and  scale 
of  training  activities.  Reasons  for  including  individual  and  course- 
related  information  have  been  mentioned  above,  and  the  unit  files  could 
help  in  assessing  the  effectiveness  of  1ST  and  on-the-job  training  with 
respect  to  job  and  unit  performance.  An  additional  file,  the  end-item 
file,  contains  data  on  the  occupation  profile  of  personnel  needed  to 
operate  and  maintain  items  in  the  defense  inventory.  This  file  can  be 
used  to  gauge  the  manning  and  training  requirements  of  new  weapons 
systems.  The  versatility  of  the  proposed  files  can  be  best  illustrated 
by  example,  and  the  next  section  discusses  a  number  of  possible 
applications . 


*A  good  deal  of  data  on  students  and  military  instructors  can  be 
obtained  from  DMDC  personnel  files;  for  civilian  instructors  it  would  be 
necessary  to  query  federal  civilian  personnel  files.  TPDC  might  keep  in 
house  a  subset  of  data  from  the  personnel  records  maintained  at  DMDC. 
Information  such  as  age,  race,  sex,  education,  AFQT,  years  of  service, 
and  paygrade  might  be  kept  at  TPDC,  with  updates  every  six  months  or 
annually,  as  opposed  to  DMDC's  quarterly  updates.  This  would  enable 
cursory  analyses  and  tabulations  to  be  performed  while  more  specific 
personnel  data  is  being  obtained  from  DMDC. 
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Table  2 

INITIAL  SKILL  TRAINING  DATABASE  FILES 


A.  DATA  RELATED  TO  INDIVIDUALS  (identifiers:  SSN,  ISTN,  MOS,  UIC) 

1.  PERSONNEL  RECORD 

AFQT 

Demographics 

Attrition 

Promotion  history 

Term  of  enlistment 

Reenlistment 

Occupation 

Length  of  service 

Dates  for  MOS  and  unit  changes 

2.  PERFORMANCE  IN  1ST 

Test  scores 
Rank  in  class 
Attrition 
Pass/fail  status 
Recycle  status 

3.  PERFORMANCE  ON  THE  JOB 

Promotion 

Performance  reviews 
Supervisor  rating 
Skill  qualification  test  scores 
Dates  for  tests  and  review 

4.  POST-SERVICE  EXPERIENCE 

Earnings 

Employment  history 

Occupation 

Location 

B.  DATA  RELATED  TO  COURSES  (identifiers:  ISTN,  MOS,  time  period) 

1.  OCCUPATION  DESCRIPTION 

Skill  codes  (MOS,  DOT,  DOD) 

Required  skills 
Eligibility  criteria 

2.  COURSE  DESCRIPTION 

Content 
Skills  taught 
Prerequisites 
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Table  2 --Continued 


3.  CLASS  INFORMATION 

Date 

Equipment 
Number  entering 
Location 

Instructor  characteristics 
Number  graduating 
Length 

Instructional  methods 

4.  TESTS  AND  GRADING  STANDARDS 

Test  type 
Content 
Passing  scores 
Administrative  procedures 

C.  DATA  RELATED  TO  OCCUPATION  (identifiers:  MOS ,  time  period) 

1 .  OCCUPATION  DESCRIPTION 

Skill  codes 

Promotion  requirements 
Required  skills 
Grade  distribution 
Duties 

2.  ELIGIBILITY  REQUIREMENTS 

ASVAB  cutoff  scores 
Physical  requirements 

3.  END  ITEM  FILE 

MOS  profile 

Number  of  personnel/MOS 

D.  DATA  RELATED  TO  UNITS  (identifiers:  UIC,  MOS,  time  period) 

1.  UNIT  MISSION  AND  LOCATION 

Required  tasks 
CONUS/NONCONUS 
Size  of  base 
Personnel  requirements 

2.  UNIT  READINESS 

Exercise  scores 
Maintenance  performance 
Staffing  status 
Equipment  status 
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Table  2--Continued 


3.  EQUIPMENT 

Type 

Quality 

Whether  at  authorized  levels 
Age 

4.  PACE  OF  OPERATIONS 

Average  workload 
Amount  of  downtime 

Amount  of  time  performing  assigned  tasks 

E.  DATA  RELATED  TO  TRAINING  RESOURCES  (identifiers:  MOS,  ISTN, 
time  period) 

1.  PERSONNEL  COSTS 

Student's  pay 
Relocation 
Instructor's  pay 
Accession 

Time  spent  in  on-the-job  training 
(not  readily  available;  would  have 
to  be  estimated) 

2.  EQUIPMENT  COSTS 

Acquisition 

Maintenance 

Depreciation 

3 .  COURSE  COSTS 

Design  costs 
Course  materials 

4.  BASE  OPERATIONS 
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III.  APPLICATIONS  FOR  AN  INITIAL  SKILL  TRAINING  DATABASE 


In  the  previous  section,  we  described  the  kind  of  information 
desirable  in  the  TPDC  databank.  That  information  fits  into  five  broad 
categories  or  possible  1ST  data  files  related  to:  (1)  the  individual, 
(2)  courses,  (3)  occupation,  (4)  unit,  and  (5)  training  resource*.  The 
1ST  database  is  designed  to  contain  information  on  the  course,  students, 
and  training  outcomes.  Variables  relating  to  equipment,  training 
devices,  instructors,  curriculum,  tests,  and  student  characteristics  can 
be  combined  to  analyze  the  effects  of  historical  variation  in  those 
variables  on  test  scores  and,  potentially,  subsequent  outcomes  on  the 
job.  In  other  instances,  experimentation  may  be  necessary  to  properly 
evaluate  changes  in  training  programs  and  policy.  The  data  collection 
mechanisms  established  for  the  1ST  database  would  make  the 
experimentation  process  simpler  and  cheaper  since  fewer  special  data 
collection  efforts  would  be  needed  beyond  those  already  in  place. 

The  1ST  database  can  support  analyses  other  than  those  directly 
related  to  the  evaluation  training  programs  and  policies.  Indeed,  there 
are  applications  for  the  database  that  can  aid  training  management,  help 
develop  future  training  requirements,  and  provide  insights  helpful  to 
recruiters  and  personnel  managers.  To  illustrate  the  varied  uses  of  the 
1ST  database,  we  have  chosen  to  focus  on  several  areas  where,  in 
addition  to  the  evaluation  of  training  program  effectiveness,  the 
database  could  play  an  important  role  in  helping  the  services  and  DoD 
address  critical  training  problems.1  These  areas  include: 


lWhereas  the  1ST  database  is  designed  to  focus  on  issues  and 
problems  related  to  initial  skill  training  and  first-term  performance, 
the  database  can  also  provide  useful  information  for  examining  broader 
issues  such  as  the  impact  of  different  career  mixes  on  the  capability  of 
the  unit  to  meet  its  mission.  These  broader  issues  include  factors  that 
go  beyond  the  data  suggested  for  the  1ST  database,  and  while  the 
database  can  supply  input  to  analyses  of  those  issues,  we  have  chosen  to 
confine  our  illustration  of  applications  to  those  specifically  related 
to  1ST  performance. 
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•  Determining  training  loads 

•  Matching  weapons  system  acquisition  and  training  requirements 

•  Managing  skills  shortages 

•  Selecting  personnel  into  military  occupations 

•  Evaluating  the  individual's  return  to  military  training 


1  Table  3  is  a  guide  to  the  following  discussion  of  applications. 

The  table  illustrates  the  interface  between  the  1ST  data  files  and  the 
£  five  applications  and  summarizes  which  data  files  could  support  each 

(  application.  As  can  be  seen,  each  of  the  data  files  would  be  used  in 

nearly  all  of  the  applications,  reflecting  the  wide  variety  of  research 
(  topics  to  which  a  single  file  contributes. 

Table  3 

APPLICATIONS  FOR  1ST  DATABASE  FILES 


Determination 

of 

Training  Load 

Weapons 

System 

Acquisition 

Shortage 

of 

Manpower 
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Individual ' s 
Return  to 
Training 

Data 
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related 

individual 

X 

X 

X 
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Data 

to 
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X 
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X 

X 
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related 
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Data 

to 
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unit 

X 

X 
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Data 
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costs 

X 
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X 

X 
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DETERMINING  TRAINING  LOADS  FOR  BUDGETING 

Each  year  the  Office  of  Secretary  of  Defense  issues  a  Military 
Manpower  Training  Report  giving  information  that  supports  the  Department 
of  Defense  request  for  authorization  of  average  military  student 
training  loads  and,  indirectly,  the  training  budget.  The  training  load 
is  defined  as  the  number  of  manyears  of  training  provided  during  the 
fiscal  year.  Roughly,  it  equals  the  average  number  of  students  in  a 
course  times  course  length  expressed  as  a  fraction  of  a  year.  The 
r  enlisted  1ST  training  load  was  estimated  to  be  82,730  in  FY  1987,  which 

*  is  comparable  to  the  amount  of  instruction  time  provided  by  three  35,000 

student  universities  operating  on  a  nine  month  academic  year.  In  a 
>  sense,  the  process  of  determining  training  load  begins  with  a 

^  determination  of  the  requirement  for  newly  trained  personnel,  indicating 

how  many  graduates  the  course  should  produce,  and  the  construction  of 
the  course  itself,  implying  course  duration.  Most  courses  are  9  to  16 
t  weeks  long,  the  average  being  11  to  12  weeks.  The  next  ingredient 

needed  is  the  attrition  rate.  Given  the  attrition  rate  and  desired 
number  of  graduates,  one  computes  the  necessary  number  of  starting 
students;  the  average  number  of  students,  then,  equals  (approximately) 
the  simple  average  of  starters  and  graduates.  Training  load  (average 
number  of  students  x  course  length)  is  computed  for  each  1ST  course  and 
aggregated  to  obtain  the  total  training  load.  The  services  offer  over 
1100  1ST  courses  each  year  for  enlisted  personnel. 

What  figure  should  be  used  for  a  course's  attrition  rate?  Current 
practice  employs  the  average  rate  observed  in  recent  classes,  but  more 
accurate  estimation  seems  possible  by  constructing  a  model  of  1ST 
attrition.  The  model  could  analyze  the  effect  of  factors  on  course 
attrition,  including  student  socioeconomic  background  and  aptitudes, 

.  occupation  selection  criteria,  instructor  characteristics,  course 

length,  and  tests  and  grading  criteria.  If  there  were  changes  in  the 
curriculum,  equipment,  and  teaching  aids,  they  too  might  affect 
attrition,  as  might  the  number  of  students  being  recycled  through  the 
course.2  The  purpose  of  the  model  would  be  twofold.  In  the  short  run 
k  __________ 

furthermore ,  course  attrition  must  be  viewed  in  the  context  of 
service  policy  toward  attrition  and  current  manning  pressure  to  increase 


22 


it  would  assist  in  the  estimation  of  training  loads,  depending  on  the 
way  courses  are  structured  and  taught.  In  the  long  run  it  would 
contribute  a  deeper  understanding  of  the  variables  influencing  course 
attrition  and  possibly  suggest  changes  that  would  significantly  lower 
the  rate  while  maintaining  course  content  and  quality. 

In  the  very  short  run,  gains  from  more  accurate  attrition  forecasts 
might  be  minor.  This  is  because  historical  extrapolation--the  use  of 
recent  attrition  rates  to  forecast  near-term  future  rates--is  reasonably 
accurate  when  the  underlying  structure  is  constant.  That  is,  if 
eligibility  requirements,  instructors,  equipment,  length,  and  course 
content  remain  the  same,  then  there  should  be  little  change  in  course 
attrition.  But  in  the  longer  run,  naturally  occurring  variations  in 
these  factors  would  affect  attrition,  and  data  on  such  changes  would 
provide  a  foundation  for  their  quantitative  analysis.  Such  analysis  and 
forecasts  seem  most  important  for  hard-to-fill  occupations  and  for 
occupations  with  long,  complex  training  where  up  to  a  third  of  the 
students  might  not  complete  the  class.1  In  comparison,  the  overall  1ST 
attrition  rate  for  enlisted  personnel  is  7.5  percent.* 

The  vast  majority  of  existing  attrition  studies  are  limited  to 
information  on  student  descriptors  and  aptitudes  such  as  age,  sex,  race, 
high  school  graduation,  and  aptitude  scores  from  the  Armed  Services 
Vocational  Aptitude  Battery  (ASVAB).  Probably  the  strongest  and  most 
consistent  finding  of  these  studies  is  that  the  attrition  rate  of  high 
school  non-graduates  is  twice  as  high  as  that  of  graduates.  The 
contribution  of  the  other  variables  is  usually  minor,  and  predicting 
attrition  therefore  begins  simply  with  knowing  the  percentage  of 
students  who  are  high  school  graduates.  Ironically,  the  recruiting 
successes  of  the  1980s  have  diminished  the  value  of  this  approach:  In 
recent  years  about  90  percent  of  the  recruits  have  been  graduates,  but 
attrition  rates  have  fallen  much  less  than  has  been  expected. 

or  reduce  the  number  of  first-term  personnel  as  a  buffer  to  meet  and  not 
exceed  endstrength  goals. 

3 Military  Manpower  Training  Report  FY1987 ,  p.  V-7. 

* Ibid .,  computed  from  Table  V-2,  p.  V-4. 
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Further  insight  into  the  causes  of  attrition  will  not  be  gained 
unless  more  comprehensive  data  become  available  on  occupation  selection 
criteria,  course-related  information,  and  attrition  policy.  (See  Table 
3  for  the  data  file  types  used  for  determining  training  loads.)  More 
comprehensive  data  require  the  development  of  several  data  files, 
including  an  occupation  selection  file  and  course  files.  An  occupation 
selection  file  would  give  eligibility  information  for  selection  into  an 
occupation.  For  example,  if  selection  criteria  were  tightened  over 
time,  the  course's  attrition  might  decline  as  more  studerts  passed  the 
tests.  Course  files  would  describe  the  class  and  its  equipment, 
training  devices,  instructors,  and  testing.  The  course  descriptors 
would  specify  course  objectives,  skills  taught,  and  duration. 

Information  on  skills  taught  could  be  compared  with  an  occupation  file 
describing  the  duties  and  the  necessary  skills  pertinent  to  each 
occupation.  A  comparison  between  the  occupation  skills  and  course 
skills  would  differentiate  those  skills  taught  during  the  course  from 
those  presumably  learned  on  the  job. 

The  file  on  course  equipment  would  show  the  specific  kind  of 
equipment  used  in  training.  A  unit  file,  cataloging  the  equipment  in 
the  unit  to  which  the  student  is  subsequently  assigned,  could  be  used  to 
compare  whether  training  equipment  was  the  same  as  encountered  in  the 
field.  The  file  on  training  devices--teacher  aids  and  simulators-- 
would  document  any  special  items  used  during  instruction;  for  example, 
changes  in  the  presence  of  simulators  could  affect  course  attrition. 

The  instructor  file  would  contain  information  about  the  instructor's 
education,  teaching  experience  (for  instance,  the  number  of  times  the 
person  has  taught  the  course),  and  SSN,  permitting  a  tie  to  the 
personnel  record.  Such  linkages  would  enable  analyses  of  how  instructor 
quality  affects  course  attrition.  The  test  file  would  contain  a 
description  of  the  testing  regime;  there  might  be  ten  weekly  written 
tests,  three  hands-on  tests,  and  a  final,  and  perhaps  all  would  be 
graded  pass/fail.  The  tests  themselves  might  be  (required  to  be)  kept 
on  file  at  the  course  site. 
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Another  useful  file  would  contain  information  on  service  attrition 
policies  and  practices  over  time.  Most  attrition  analyses  focus  on  the 
relationship  between  the  individual's  characteristics  and  attrition 
behavior.  However,  equally  important  are  service  attrition  policies. 

The  most  striking  evidence  of  this  is  the  fact  that  training  attrition 
rates  did  not  fall  when  the  percentage  of  high-quality  recruits  (i.e., 
those  who  are  high  school  graduates  and  in  the  50th  percentile  or  higher 
of  the  AFQT)  increased  dramatically  in  the  1980s.  Previous  attrition 
studies  showed  a  strong  positive  relationship  between  attrition  and  non- 
high  school  graduation  status.  The  expected  result  of  a  large  increase 
in  high  school  graduates  among  the  recruit  population  would  be  lower 
attrition  rates  for  such  cohorts.  However,  this  was  not  the  case. 
Service  attrition  practices  apparently  played  a  major  role  in 
maintaining  attrition  rates  at  or  above  the  rates  when  the  proportion  of 
high  school  graduates  was  lower.5  The  Army,  alarmed  by  the  continued 
high  attrition  among  high  quality  recruits,  instituted  a  study  which 
recommended  a  new  training  attrition  program  to  lower  attrition  through 
such  actions  as  retraining  of  drill  instructors;  better  evaluation, 
counseling,  and  remediation  efforts;  remedial  physical  conditioning;  and 
an  emphasis  on  positive  leadership  encouraging  reasonable  progress  and 
standards  for  recruits.  Six-month  attrition  fell  sharply  after  the  Army 
began  to  institute  the  new  recommendations.  This  program  provided 
evidence  that  service  policies  as  well  as  recruit  quality  have  an 
important  effect  on  attrition  rates. 

The  importance  of  service  attrition  policies  suggests  that  thought 
be  given  to  the  bookkeeping  conventions  for  attrition.  The  main 
concerns  of  such  bookkeeping  are  to  distinguish  course  attrition  from 
service  attrition  and  improve  the  way  service  attrition  is  recorded. 
Course  attrition  need  not  lead  to  service  attrition;  the  student  might 

sBuddin  (forthcoming)  found  substantial  differences  in  attrition 
rates  among  training  bases  and  among  entering  cohorts  for  recruits  of 
comparable  quality.  Thus,  while  recruit  characteristics  may  be  used  to 
rank  prospective  recruits  according  to  their  potential  risk  of 
attrition,  service  policies  and  practices  have  a  critical  effect  on  the 
determinat ion  of  the  actual  attrition  level. 
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be  recycled  through  the  course.  Recycling,  as  well  as  performance  in 
the  course,  could  be  tracked  by  a  student  training  file,  i.e.,  the 
"individual  performance  in  1ST"  file  listed  in  Table  2.  However,  course 
attrition  and  service  attrition  are  often  one  and  the  same,  if  only 
because  the  latter  necessitates  the  former,  so  the  Interservice 
Separation  codes  used  to  designate  a  reason  for  attrition  should  be  able 
to  discriminate  poor  classroom  performance  as  a  reason  for  service 
attrition  from  other  reasons.  Otherwise,  it  is  impossible  to  tell 
whether  an  increase  in  course  attrition  derives  from  changes  internal  or 
external  to  the  course.  At  present  there  are  70  or  so  assignable 
reasons  for  discharge,  which  may  be  too  many.  Categories  should 
illuminate  the  causes  of  attrition;  when  there  are  too  many  the  reasons 
for  discharge  tend  to  obscure  rather  than  clarify  the  attrition  picture. 
Too  many  codes  creates  ambiguity  in  designating  "the"  reason  for 
discharge,  so  creating  a  few  broad  categories  seems  preferable  (e.g., 
failed  training,  discipline,  health  status,  attitude,  legal 
infraction).6  Effecting  such  a  change  will  require  training  and 
procedures  for  uniform  implementation  of  the  categories  across  the 
services,  a  considerable  undertaking. 

Aside  from  helping  to  improve  estimates  of  training  loads,  a  better 
model  of  training  attrition  would  could  help  recruiters.  Reducing 
training  attrition  through  a  clearer  understanding  of  its  determinants 
will  help  hold  down  accession  requirements.  In  addition,  accurate 
forecasts  of  training  attrition  foster  accurate  forecasts  of  accession 
requirement  forecasts,  aiding  commanders  in  determining  quotas  for  their 
individual  recruiters.  Also,  by  knowing  the  individual  factors  that 
correlate  with  training  attrition,  the  recruiter  can  better  judge  which 
recruiting  prospects  are  likely  to  complete  training  and  which  ones  are 
not . 


sSubcategories  could  be  designated  within  each  of  the  broad 
categories,  but  again,  ambiguity  of  assignment  should  be  avoided. 
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MATCHING  WEAPONS  SYSTEM  ACQUISITION  AND  TRAINING  REQUIREMENTS 

A  prominent  element  in  the  debate  over  weapon  systems  acquisition 
is  the  role  of  manpower.  There  is  fear  that  new  weapons  will  be  too 
complicated  to  operate  and  maintain,  so  their  potentially  greater 
military  capability  will  be  reduced  or  nullified  by  inordinate  personnel 
demands.  In  light  of  growing  concern,  the  Secretary  of  Defense  must  now 
present  manpower  estimates  tied  to  any  new  weapons  system  to  Congress 
before  full-scale  engineering  or  production  and  deployment  of  that 
system  can  be  approved.7  These  estimates  focus  on  whether  there  will  be 
enough  properly  trained  people  available  to  operate  and  maintain  the 
system  and  what  the  cost  will  be,  both  to  achieve  and  to  maintain  it. 

Because  manpower  becomes  a  sizable  factor  in  the  lifecycle  cost  of  a 
weapon  system,  it  is  argued  that  greater  attention  to  manning 
requirements  during  design  and  development  will  result  in  lower  system 
cost  without  degrading  system  performance,  reliability,  and 
maintainability.  The  directive  to  include  manning  considerations  in  the 
early  phases  of  the  acquisition  process  has  proved  difficult  to 
implement  because  manning  requirements  are  inherently  imprecise  so  long 
as  weapon  technology  and  mission  are  in  flux.  Still,  if  alternative  or 
existing  designs  could  be  rapidly  translated  into  a  reasonable  range  of 
manning  requirements,  the  opportunity  would  be  improved  for  using 
manning  requirements  to  determine  or  reconceive  design. 

The  1ST  database,  as  described  in  Sec.  II,  could  lend  support  to 
the  individual  services  and  to  OSD  in  estimating  the  requirements  for  a 
new  system.  In  its  "Footprint”  project,  TPDC  suggests  that  first-round 
manning  estimates  could  be  readily  derived  in  many  cases.  The  TPDC 
approach,  which  is  the  basis  for  the  present  discussion,  uses  an 
existing  file  of  "end  items"  and  their  occupation  profile  as  the  basis 
for  interpolating  manning  and  training  requirements  from  present  systems 

7The  MANPRINT  system  is  the  Army's  response  to  the  Congressional 
requirement.  It  is  designed  to  be  a  comprehensive  management  and 
technical  program  to  improve  total  system  performance  by  the  continuous 
integration  of  human  factors  engineering,  manpower,  training,  system 
safety,  and  health  hazard  considerations  throughout  the  materiel 
development  and  acquisition  process. 
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to  proposed  systems.  Every  end  item  in  the  current  defense  inventory- 
jeep,  bazooka,  radio,  missile,  field  hospital--has  a  profile  showing  the 
number  of  personnel  by  occupation  required  to  operate  and  maintain  it, 
and  many  of  the  end  items  in  a  new  weapon  system  may  be  similar  to 
existing  items.  If  so,  the  occupation  profile  and  manning  requirements 
of  the  new  system  can  be  approximated,  and  they  in  turn  can  be  related 
to  training  requirements.  For  example,  suppose  a  new  system  requires  an 
end  item  similar  to  a  combination  of  two  existing  end  items;  the  planner 
must  designate  which  existing  end  items  will  serve  as  the  basis  of 
comparison  and  what  weight  should  be  given  to  each.  This  leads  to  a 
rough  occupation  profile  of  the  new  item,  say  half  a  person  in 
occupation  "a"  and  a  full  person  in  occupation  "b".  As  mentioned  above, 
for  each  occupation  there  is  also  a  file  describing  its  tasks  and  skills 
and  a  course  file  listing  those  skills  taught  in  1ST.®  Thus,  this 
approach  yields  information  on  the  required  increase  in  trained 
personnel  by  occupation,  the  skills  they  possess,  and  the  impact  on 
training  load. 

A  more  complicated  example  can  easily  be  envisioned.  Using  the 
previous  example  as  a  first  round,  the  planner  could  refine  the 
occupational  requirements  by  specifying  the  subset  of  skills  that  would 
be  needed  from  each  occupation.  This  might  entail  restructuring 
occupations  "a"  and  "b"  into  a  hybrid  "c"  and  the  development  of  a  new 
course  that  taught  the  relevant  skills  from  the  courses  for  "a"  and  "b" . 
Of  course,  skills  not  taught  during  1ST  would  have  to  be  learned  on  the 
job,  but  the  more  skills  taught  during  1ST,  the  longer  the  course  and 
the.  greater  its  cost  (including  student  pay).  Possible  refinements 
relate  to  teaching  aids,  training  devices  and  simulators,  the  equipment 
that  would  be  needed  for  the  new  course,  and  the  specification  of 
occupation  selection  criteria.  Again,  if  the  new  course  mainly  taught 
skills  already  taught,  then  current  experience  would  be  a  good  guide  to 
new  training  requirements. 

®There  are  other  kinds  of  formal  training  that  could  be  considered, 
the  most  common  being  Skill  Progression  Training.  It  is  most  frequently 
given  after  servicemembers  have  gained  experience  through  actual  work  in 
their  specialty. 
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These  examples  suggest  the  usefulness  of  the  end- item/occupation 
profile  information  coupled  with  files  describing  occupations,  skills 
required,  and  skills  taught  in  1ST.  While  greater  flexibility  results 
from  the  possibility  of  restructuring  occupations,  courses,  and 
selection  criteria,  the  more  detailed  projections  come  at  a  higher  cost. 
They  are  perhaps  better  done  as  second-round  estimates  of  manning 
requirements  for  variants  of  a  new  system. 

The  cost  of  alternative  manning  and  training  options  should  also  be 
computed.  To  do  so,  a  cost  estimation  methodology  must  be  specified  and 
files  created  containing  the  relevant  resource  information.  If  the 
costs  of  trained  personnel  for  a  given  system  are  high,  planners  might 
want  to  respecify  system  manning  requirements.  The  costs  will  depend  on 
direct  1ST  resources  such  as  instructors,  courseware,  equipment,  and 
space,  and  also  on  the  quantity  and  quality  of  accessions  required.  If 
the  requirements  of  the  new  system  parallel  those  of  the  system  being 
phased  out,  perhaps  few  new  resources  would  be  needed;  existing 
resources  would  be  shifted  around  to  meet  the  new  requirements. 

MANAGING  SKILL  SHORTAGES 

Skill  shortages  occur  by  occupation  and  are  inherently  transitory 
because  the  system  responds,  eventually,  to  correct  them.  Shortages 
ought  to  be  measured  in  terms  of  employment,  so,  for  example,  a  shortage 
may  exist  at  a  given  level  of  military  compensation  but  disappear  when  a 
bonus  or  benefit  is  introduced.  Still,  the  response  may  be  slow  or 
inadequate  and  there  may  be  a  lag  before  it  takes  effect,  all  of  which 
prolong  the  shortage.  Shortages  might  arise  from  unanticipated 
increases  in  manning  requirements,  high  attrition,  or  low  reenlistment. 
Depending  on  the  causes,  shortages  may  occur  in  a  single  skill  or  be  far 
broader.  In  the  late  1970s,  for  instance,  military  pay  fell  below 
civilian  pay  at  a  time  when  the  unemployment  rate  was  low,  leading  to 
lower  retention  overall  which  in  turn  caused  higher  enlistment 
requirements.  Although  recruiters  often  managed  to  achieve  their 
enlistment  targets,  they  enlisted  lower  quality  recruits  who  were  far 
harder  to  train. 
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The  usual  adjustments  to  a  shortage  occur  through  recruiting, 
retention,  and  retraining.3  From  the  perspective  of  1ST,  recruiting  and 
retraining  have  an  immediate  impact  on  training  activities:  Training 
must  be  provided  to  the  additional  recruits  and  retrainees  channeled 
into  the  shortage  occupations.  Extra  training  leads  to  questions  about 
how  readily  a  particular  1ST  course  can  expanded,  or,  if  the  shortage 
is  widespread,  how  readily  1ST  courses  can  be  expanded  in  general. 

Other  adjustments  may  also  be  made,  such  as  reconfiguring  units  to 
substitute  higher  or  lower  skilled  personnel  for  the  short  skill,  or 
adding  labor-saving  equipment  (e.g.,  maintenance  aids),  but  such 
adjustments  take  longer  to  implement  and  may  require  1ST  course 
restructuring. 

These  questions  raise  details  that  are  a  routine  part  of  the 
services'  planning  and  programming  of  training  resources.  However,  the 
services'  responses  and  the  conditions  affecting  those  responses  are 
virtually  unrecorded.  It  is,  therefore,  difficult  to  benefit  from  that 
experience.  As  a  positive  step,  TPDC  could  acquire  information  on  the 
planned  number  of  courses  of  a  given  kind  of  1ST  offered  each  year,  the 
number  actually  given,  the  planned  and  actual  numbers  of  students 
(entrants  and  graduates),  the  training  equipment  available  for  the 
course,  and  an  indication  of  equipment  shortages.  As  these  data 
accumulated,  a  record  of  the  range  of  1ST  responsiveness  would  emerge. 
Preferably,  the  data  could  be  tied  to  a  file  indicating  when  1ST  was 
responding  to  a  skill  shortage,  helping  document  how  specific  shortages 
were  eliminated. 

To  assess  1ST  flexibility  it  would  also  be  useful  to  track  the 
response  to  anticipated  and  unanticipated  variations  in  the  training 
load.  Under  current  practice,  the  supply  of  training  seats  (i.e.,  the 
rate  at  which  a  course  is  offered)  is  governed  by  the  objective  of 
maintaining  a  fairly  steady  flow  of  newly  trained  personnel  as 
replacements  for  losses  due  to  attrition  and  non-reenlistment.  Since 
the  supply  of  applicants  is  seasonal,  with  peaks  during  the  summer 

’The  recruiting  and  retention  paths  may  employ  bonuses  and  benefits 
as  well  as  greater  effort  by  recruiters  and  reenlistment  counselors. 
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months  and  at  the  turn  of  the  year,  smoothing  often  means  longer  waits 
in  the  Delayed  Entry  Program  (DEP)  before  the  enlistee  accesses  and 
begins  training.  There  is  also  some  variation  in  1ST  offerings  to  fit 
the  pattern.  The  file  on  course  offerings  mentioned  above  would  depict 
the  1ST  pattern  and  show  the  peak  student  load  accommodated.  That  would 
presumably  indicate  the  manageable  load  if  a  short-run  expansion  of  the 
force  were  required. 

Recruiting  success  in  overcoming  shortages  might  depend  on  the 
availability  of  training.  An  interested  recruit  might  choose  not  to 
enlist  if  the  wait  until  training  (DEP  length)  were  long,  say  six  months 
instead  of  two.  As  it  is,  the  most  highly  sought  occupations  have  long 
DEP  lengths  and  shortages  are  rarely  a  problem.  In  contrast,  the  effect 
of  greater  training  availability  on  enlistment  might  prove  important  in 
less  popular  occupations.  Experimentation  might  be  required  to 
determine  this,  and  data  on  the  flexibility  of  IST--variation  in  courses 
per  quarter,  students  per  course--could  form  a  valuable  baseline. 

In  dealing  with  skill  shortages,  the  1ST  data  are  not  necessarily 
restricted  to  the  retraining  and  recruiting  options.  1ST  data  can  also 
help  with  retention:  information  on  training  resources  and  costs  in  the 
1ST  database  might  aid  the  reenlistment  bonus  allocation  process.  Given 
that  reenlistment  bonus  multiples  are  in  part  determined  by  training 
costs,  1ST  training  cost  data  might  enable  more  accurate  methods  of 
Selective  Reenlistment  Bonus  (SRB)  allocation  in  times  of  skill 
shortages . 

Finally,  files  containing  information  on  course  attributes  would  be 
helpful  in  planning  for  shortages  consequent  to  a  mobilization. 

Depending  on  a  course's  resource  requirements  for  instructors, 
equipment,  teaching  aids,  facilities,  and  duration,  and  its  capacity  to 
increase  its  student  load,  the  course  might  or  might  not  represent  a 
bottleneck  in  a  given  scenario. 
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SELECTING  PERSONNEL  INTO  MILITARY  OCCUPATIONS 

Training  data  can  help  address  the  questions  of  whether  the 
allocation  of  recruits  across  military  occupations  is  efficient  and 
whether  the  level  of  efficiency  attained  is  sensitive  to  moderate 
changes  in  the  allocation.  These  questions  generally  arise  in  a  context 
that  assumes  the  set  of  occupations  and  their  courses  is  given.  The 
problem  is  therefore  similar  to  a  classical  matching  problem:  find  an 
allocation  of  personnel  that  is  superior  or  preferable  to  other 
allocations  in  producing  the  highest  expected  value  to  the  military. 

This  problem,  which  can  be  stated  and  solved  formally,  becomes  difficult 
to  solve  in  practice  because  a  solution  cannot  be  achieved  without  an 
explicit  measure  of  the  value  to  the  military  of  placing  a  given 
individual  in  a  specific  occupation.  Lacking  an  explicit  valuation,  the 
military  approaches  the  matching  problem  indirectly.  Its  approach  can 
be  characterized  by  three  objectives: 

1.  Make  sure  every  occupation  is  fully  manned; 

2.  Make  sure  every  occupation  gets  a  reasonably  representative 
distribution  of  recruits  by  "quality;" 

3.  Minimize  the  chance  that  anybody  assigned  to  an  occupation  will 
perform  inadequately,  i.e.,  be  unable  to  complete  training  or 
perform  required  tasks. 

Recruiters  employ  enlistment  incentives  to  accomplish  the  first  two 
of  these  objectives.  In  conjunction,  the  third  objective  is  addressed 
by  an  occupational  assignment  system.  Automated  occupation  assignment 
systems  review  the  current  manning  status  and  quality  distribution  of 
occupations  and  recommend,  via  a  CRT  screen,  the  "best"  occupational 
choices  for  a  recruit  given  his  aptitudes.  To  make  this  recommendation, 
programmed  information  relates  aptitudes  to  prcbable  completion  of 
training.  Personnel  with  low  aptitude  scores  could  be  deemed  unlikely 
to  complete  training,  and  hence  would  be  denied  eligibility  to  enter  an 
occupation.  The  cutoff  aptitude  scores  for  an  occupation  are  based  on 
past  experience,  specifically,  the  statistical  relationship  between 
recruits'  aptitude  scores  and  completion  of  1ST. 
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The  services  conduct  their  own  analyses  of  the  relationship  between 
aptitude  and  training  completion,  yet  TPDC  files  could  complement  their 
efforts  by  promoting  an  exchange  of  methods  and  information  across  the 
services  and  to  OSD,  and  possibly  enlarging  the  scope  of  analysis. 
Relevant  files  would  include  the  1ST  and  job  performance  files,  course 
files,  and  unit  files  mentioned  in  Sec.  II,  as  well  as  DMDC  personnel 
records.  It  would  be  important  to  distinguish  between  attrition  from 
the  service  stemming  from  poor  performance  in  the  course  and  attrition 
from  other  reasons  unrelated  to  the  course.  The  simplest  approach, 
then,  would  be  to  relate  passing  status  to  aptitudes.  But  a  more 
comprehensive  approach  would  include  educational  attainment  and 
physiologic  variables  (height,  weight)  and  control  for  changes  in  course 
content,  test  content,  grading  standards,  equipment,  teaching  aids,  and 
service  attrition  policy.  Moreover,  the  statistical  framework  would 
distinguish  course-related  attrition  from  other  attrition--both  kinds  of 
attrition  are  relevant  to  predicting  the  number  of  trained  personnel, 
but  for  different  reasons.  If  other  attrition  were  high,  it  might  be 
because  the  course  drew  attrition-prone  recruits  (those  with  low 
education,  for  example)  or  because  attrition  was  generally  high.  By 
careful  assessment  of  these  other  factors,  a  clearer  picture  would 
emerge  of  the  correlates  of  success  in  the  course  per  se.  Further, 
controlling  for  the  other  factors  facilitates  the  year-to-year  and 
service-to-service  comparability  of  relationships  between  aptitudes  and 
training  success. 

A  joint  service  effort  has  been  under  way  for  five  years  to  link 
job  performance  to  eligibility  criteria.10  This  effort  represents  an 
expansion  in  scope  from  training  success  to  performance  on  the  job, 
which  many  consider  a  more  meaningful  criterion  for  determining 
eligibility.  The  effort  is  directed  to  a  subset  of  military  occupations 

,0The  Joint-Service  Job  Performance  Measurement/Enlistment 
Standards  Project  was  undertaken  in  response  to  a  Congressional  mandate 
urging  the  military  services  to  establish  a  research  and  development 
program  to  link  enlistment  standards  with  job  performance.  Annual 
reports  from  this  project  have  been  submitted  to  Congress  since  1982. 

In  addition,  the  Army  has  a  major  project  in  this  area. 
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and  seeks  to  develop  suitable  performance  measures  and  tests.  As  the 
effort  matures,  it  should  provide  data  for  analysis,  together  with  the 
data  described  above.  The  results  may  reveal  whether  narrow  or  wide 
changes  in  cutoff  scores  degrade  training  success,  i.e.,  graduation  from 
1ST.  The  analysis  may  also  show  how  variations  in  the  average  quality 
of  personnel  entering  an  occupation  affect  training  and  job  performance 
on  average.  If  so,  that  would  constitute  a  significant  broadening  of 
the  third  objective,  help  to  minimize  the  chance  that  anybody  assigned 
to  an  occupation  would  perform  inadequately.  Rather  than  just 
minimizing  the  chance  of  failing  training  or  performing  inadequately, 
the  emphasis  would  expand  to  how  well  those  admitted  are  expected  to 
do.11 

Still,  the  opportunity  for  such  analysis  comes  at  a  cost  of  greater 
data  requirements  and  analytical  complexity.  Just  as  training  analyses 
should  control  for  course-related  variables  and  attrition,  analyses  of 
job  performance  require  data  on  on-the-job  training,  which  may  depend  on 
time  in  the  unit,  its  pace  of  operations,  and  its  equipment  and  manning, 
and  on  factors  affecting  attrition  from  the  service. 

EVALUATING  THE  INDIVIDUAL’S  RETURN  TO  MILITARY  TRAINING 

The  previous  discussion  of  occupation  selection  implicitly  falls 
under  the  heading  of  the  service's  return  to  training,  which  in  general 
will  differ  from  the  individual's  return  to  training.  It  is  up  to  the 
service  to  establish  personnel  procurement,  selection,  training,  and 
assignment  policies,  including  compensation  policy,  that  permit  the 
attainment  of  its  military  capability  objectives  at  least  cost.  These 
policies,  therefore,  must  induce  a  sufficient  number  and  quality  of  men 
and  women  to  enter  the  service  and  accept  training  and  duty  assignments 
consistent  with  the  service's  objectives.  But  individuals  will  do  so 
only  if  they  believe  the  military  career  path  is  preferred  to  other 
paths  they  could  choose.  Since  only  25  or  30  of  every  100  enlistees 

1 ‘Subjective  judgment  will  still  be  required  to  decide  what  levels 
of  average  performance  are  optimal  across  occupations,  or  stated 
differently,  when  to  increase  average  performance  in  one  skill  and 
perhaps  reduce  it  in  others  from  which  personnel  are  drawn. 
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reenlist  for  a  second  term  of  service,  the  "military  career  path"  is 
frequently  a  one-term  affair  followed  by  work  or  schooling  in  the 
civilian  sector.  Holding  military  compensation  and  enlistment 
incentives  constant,  the  military's  ability  to  attract  high-quality 
enlistees  and  channel  them  into  specific  occupations  depends  on  the 
value  of  formal  and  on-the-job  training  to  the  individual.  For  many 
that  training  has  value  because  it  is  transferable  to  civilian  jobs. 
Military  occupations  that  have  a  higher  civilian  value  will  be  more 
attractive  to  recruits.12 

It  is  useful  for  the  military  to  recognize  the  value  of  training  to 
the  individual  for  several  reasons.  First,  demonstrating  that  military 
training  is  valuable  in  the  civilian  sector  substantiates  recruiting 
claims  to  that  effect  (the  truth-in-advertising  argument).13  Second, 
the  more  appealing  the  military  occupation,  the  greater  the  prospective 
supply  of  recruits  overall,  including  high-quality  recruits;  the  fewer 
enlistment  incentives  needed  to  meet  enlistment  quantity  and  quality 
targets;  the  greater  the  opportunity  to  channel  recruits  into  less 
desirable  occupations;  and  the  higher  the  eligibility  criteria  can  be 
set.  Third,  bringing  in  more  high  quality  personnel  implies  a  higher 
quality  pool  of  trained  personnel  from  which  to  draw  reenlistees.  In 
effect,  an  occupation  has  value  as  a  recruiting  instrument;  although  the 
value  may  be  perceived  differently  by  different  recruits  and,  from  a 
policy  perspective,  is  not  readily  changed  (because  course  content  is 
dictated  by  requirements  for  trained  personnel,  not  by  recruiting 
exigencies),  it  is  a  recruiting  tool  nonetheless.  It  may,  therefore,  be 
worthwhile  to  consider  whether  the  training  regimen  can  be  altered  to 
increase  the  value  of  training  to  the  individual  without  reducing  its 
value  to  the  service. 

12This  in  not  to  say  that  other  factors  are  unimportant  in  the 
enlistment  choice,  e.g.,  patriotism,  adventure,  sense  of  national  duty. 

1 ’Existing  studies  of  post-service  earnings  tend  to  be  based  on 
veterans  from  the  draft  era,  e.g.,  Danzon  (1980),  DeTray  (1981),  and 
Warner  and  Goldberg  (1987).  The  differences  in  the  composition  of 
recruits  under  the  All  Volunteer  Force  (AVF)  and  in  the  greater  average 
technical  skill  requirements  in  military  occupations  suggest  that 
studies  focusing  on  AVF  veterans  be  undertaken. 
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The  TPDC  files  needed  to  address  this  topic  are  essentially  the 
same  as  described  in  the  preceding  subsection.  They  include  the 
personnel  record,  individual  1ST  performance,  course-related 
information,  individual  job  performance,  and  unit-related  information. 
But  there  is  a  further  data  requirement--information  about  civilian 
employment  and  earnings  for  those  leaving  the  service.14  The  latter 
should  be  longitudinal  data  beginning  with  the  transition  from  the 
service  to  the  establishment  of  a  career  in  the  civilian  sector.  School 
attendance  and  educational  benefit  usage  would  also  be  relevant. 

How  these  data  would  be  collected  and  on  whom  are  unresolved 
issues.  Should  all  separatees  be  followed  or  just  a  selected  subset? 
Should  information  be  obtained  through  yearly  in-person  interviews  or 
mailed  interview  instruments?  Could  earnings  information  be  obtained 
from  other  government  sources  such  as  the  Social  Security 
Administration?  Should  selected  separatees  be  made  part  of  the  annual 
CPS  survey?  Creating  this  file  is  a  longer  range  goal  than  the  others 
discussed  in  this  paper,  but  one  to  which  future  consideration  might  be 
given. 

Because  this  analysis  entails  a  statistical  model  encompassing 
attrition  and  reenlistment  behavior,  a  side  product  would  be  results  on 
whether  attrition  is  lower  and  reenlistment  higher  for  personnel  with 
superior  performance.  Of  course,  there  must  be  some  measure  of  superior 
performance.  In  addition  to  job  performance  tests,  speed  of  promotion 
may  serve  as  an  indicator;  however,  it  has  some  problems,  for  speed  of 
promotion  is  not  only  a  function  of  the  individual's  technical 
competence,  and  leadership,  but  also  of  the  given  job,  the  service's  own 
manning  requirements,  time  in  grade,  and  time  in  service.  Thus 
individuals  with  similar  characteristics  and  performance  may  be  promoted 

l4Analyses  of  post-service  earnings  must  also  consider  what  the 
individual  would  have  earned  had  he  not  joined  the  service,  and  how  he 
compares  with  others  in  his  civilian  occupation  who  did  not  receive 
military  vocational  training.  This  requires  using  TPDC  files  in 
conjunction  with  databases  such  as  the  Current  Population  Survey  (CPS) 
and  the  National  Longitudinal  Surveys  (NLS)  to  estimate  counter factual 
earnings . 
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at  different  rates  because  they  entered  in  a  different  year  or  because 
their  occupation's  aggregate  manning  tables  called  for  different  numbers 
of,  say,  E4s  and  E5s.  Thus,  promotion  information  should  be  accompanied 
by  information  that  controls  for  these  confounding  factors. 

With  detailed  demographic  and  employment  information  on  enlistment 
prospects,  one  could  also  determine  whether  changes  in  the  content  and 
timing  of  military  training  affect  the  supply  and  quality  of  recruits. 
For  example,  if  military  training  content  were  adjusted  to  emphasize 
civilian  counterparts  that  utilize  similar  training,  would  more 
individuals  be  attracted  to  the  military  and,  if  so,  who  would  they  be? 
If  training  seats  were  more  readily  available  in  skills  with  high  return 
to  the  individual,  would  the  services  be  able  to  draw  more  recruits  from 
the  employed  sector?  Finally,  the  information  requirements  described 
here  would  be  an  excellent  foundation  for  analyzing  data  from  a 
demonstration  or  experiment  modifying  1ST  courses. 

We  have  presented  five  areas  of  application  for  an  1ST  database  and 
discussed  some  of  the  tasks  and  problems  various  data  files  could  help 
solve.  The  applications  encompass  many  issues  of  interest,  and  the 
seeming  feasibility  of  the  database  to  span  the  breadth  of  the  issues 
offers  encouragement  that  other,  emerging  issues  could  be  supported  as 
well . 
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IV.  CONSTRUCTING  AN  EFFECTIVE  TPDC  DATABASE 


The  previous  two  sections  have  discussed  files  to  include  in  the 
TPDC  database  and  applications  that  would  be  supported  by  them.1  But  we 
recognize  that  many  aspects  of  1ST  data  potentially  available  to  TPDC 
must  be  considered  fixed  in  the  short  term.  Therefore,  in  this  section 
we  discuss  an  aspect  of  database  content  and  several  aspects  of  database 
structure  that  must  be  considered  in  the  long-term  development  of  the 
1ST  database.  The  aspect  of  database  content  is  a  caution  about  using 
criterion  testing  as  a  measure  of  performance.  After  treating  that 
topic,  we  present  the  three  structural  matters: 

•  Interaction  among  service  databanks 

•  The  requirement  for  longitudinal  data 

•  The  creation  of  an  intelligent  database 

Each  aspect  is  different  and  our  comments  on  it  are  diverse,  but  the 
discussions  all  serve  the  intent  of  constructing  a  high-quality,  easily 
accessible  database. 

A  CAUTION  ABOUT  CRITERION  TESTING  AS  A  PERFORMANCE  MEASURE 

One  possible  performance  measure  is  the  results  of  criteria 
testing,  used  in  many  1ST  classes.  In  this  form  of  testing  the  student 
is  passed  or  failed  depending  on  whether  his  test  results  demonstrate 
some  acceptable  level  of  competence  in  the  skill  taught.  Since  the  test 
is  based  on  task(s)  associated  with  a  given  skill,  the  content  validity 
of  the  test  is  typically  not  a  problem.  But  criterion  testing  may  be  a 
poor  indicator  of  the  individual's  degree  of  competence.  This  method 
does  little  to  differentiate  quality  among  those  who  passed  in  terms  of, 
for  example,  how  long  it  took  to  complete  the  task  and  how  well  the 

lAlso,  as  mentioned  at  the  outset,  a  separate  effort  to  survey 
IST-related  databases  in  the  services  is  under  way;  when  completed,  this 
survey  will  be  a  valuable  companion  to  the  present  piece. 
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individual  understands  the  role  of  that  task  in  his  job  overall. 
Additionally,  the  pass/fail  grading  system  limits  the  range  of 
performance  scores,  so  statistically  it  becomes  difficult  to 
differentiate  the  better  trained  from  the  less  well  trained. 

Insufficient  variation  in  training  grades  may  make  them  statistically 
inadequate  for  linking  1ST  performance  to  future  job  performance.1 

Criterion  testing  is  very  practical  in  several  ways.  However,  its 
potential  costs  to  1ST  research  may  be  largely  unrecognized  or 
underestimated.  1ST  course  design  commonly  relies  on  the  notion  that 
mastery  of  a  skill  is  taught  by  reducing  it  to  elementary  subskills 
which  can  be  taught  step  by  step.  Each  step,  then,  is  either  learned 
competently  or  not,  and  the  steps  are  small  enough  so  that,  once 
competence  is  demonstrated,  there  may  be  little  remaining  from  which  to 
judge  the  degree  of  competence,  especially  for  skills  that  emphasize 
doing  a  task  rather  than  understanding  the  principle  behind  it.  In 
instances  when  competence  can  be  judged  through  hands-on  performance, 
criterion  testing  may  be  preferred  to  more  complex  and  harder  to 
administer  grading  schemes. 

At  least  two  steps  can  be  taken  to  overcome  the  potential  research 
disadvantages  of  criterion  testing.  First,  where  possible  the  raw 
scores  on  tests  should  be  retained;  although  this  will  frequently  be 
irrelevant  for  hands-on  testing,  it  should  apply  broadly  to  written 
testing  using  a  pass/fail  criterion.  In  addition,  the  criterion  itself 
should  be  retained,  preferably  with  an  explanation  of  why  it  was  chosen. 
Second,  when  a  student  must  demonstrate  skill  knowledge  by  hands-on  or 
computer-aided  testing,  it  seems  desirable  to  record  the  number  of 
actual  and  allowable  attempts,  or  how  long  it  took  the  individual  to 
complete  the  test.  These  steps  might  provide  more  information  about  a 
student's  degree  of  competence  than  a  single  pass/fail  indicator  when 
linked  to  job  performance  measures.  For  example,  one  could  test 
statistically  to  see  if  those  who  took  more  attempts  to  obtain  a  "pass" 

*An  Army  study  (Popelka,  1982)  on  the  utility  of  criterion  testing 
in  62  job  specialties  found  that  over  50  percent  had  sufficiently 
limited  variance  in  the  frequency  of  test  scores  applicable  to  the 
specialty  to  question  whether  meaningful  statistical  analyses  could  be 
performed. 
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do  not  perform  as  well  subsequently  on  the  job  as  those  who  needed  fewer 
tries . 
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INTERACTION  AMONG  SERVICE  DATABANKS 

Because  most  of  TPDC's  training  data  would  come  from  the  services, 
TPDC  must  work  with  them  to  devise  specifications  for  data  elements, 
periodicity,  format,  and  transmission  To  enable  cross-service 
comparisons,  the  specifications  should  evolve  toward  commonality  rather 
than  being  service  specific;  therefore,  it  is  useful  at  the  outset  for 
TPDC  to  specify  a  general  database  architecture ,  even  though  it  may  be 
revised  over  time.  In  the  long  run,  the  greater  data  commonality 
promoted  by  this  effort  should  assist  the  services  in  their  own  training 
research  and  management. 

Some  of  the  information  that  would  be  contained  in  the  data  files 
discussed  in  Sec.  II  is  now,  or  soon  will  be,  collected  by  the  services 
in  different  databases.  These  collection  efforts  simplify  the  process 
of  data  collection  for  TPDC,  since  it  can  tap  into  existing  databases 
instead  of  having  to  wait  for  the  services  to  devise  collection  schemes 
for  TPDC-desired  data.  However,  the  nature  of  the  databases  vary,  so 
protocols  for  extraction  and  transmission  of  data  to  TPDC  would  have  to 
be  devised  for  each  individual  database.  Therefore,  the  coordination 
between  the  services  and  TPDC  is  a  crucial  issue. 

The  following  list  includes  some  of  the  relevant  service  databases 
related  to  training  from  which  TPDC  might  draw  and  the  types  of 
information  they  contain. 

I.  AIMS:  Automated  Instructional  Management  System  (Army) 

The  system  is  designed  to  provide  the  U.S.  Army  Training  and 
Doctrine  Command  (TRADOC)  automation  of  mucli  of  the  training 
support  effort  required  at  each  Army  school.  Participation  in 
AIMS  by  each  installation  will  provide  TRADOC  with  automated 
reports  of  training  schedules,  training  activity,  and  training 
resource  requirements . 

The  system  keeps  track  of  individual  attributes  and  military 
career  history  data,  individual  training  experience  (including 
training  date,  course  name  and  length,  grades  and  class  standing, 
training  location),  training  course  description  (including 
prerequisites,  type  of  instructional  program,  course  length  and 
class  size  limits,  information  on  attendees  such  as  number 
entered,  number  recycled,  number  graduated,  and  the  Social 
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Security  numbers  of  graduates  which  facilitates  linking  records 
with  personnel  files),  and  training  resource  requirements  such 
as  facilities,  equipment,  instruction,  and  support  staff. 

II.  MPS:  Maintenance  Performance  System  (Army) 

The  MPS  system  was  developed  by  the  Army  Research  Institute 
as  a  training  needs  information  system  for  use  at  the  battalion 
level.  MPS  is  designed  to  be  used  by  work  supervisors  and 
training  managers  within  maintenance  skills.  The  system 
records  the  work  experience  and  training  of  each  maintenance 
technician.  Included  are  the  time  on  each  technical  task 
performed  in  the  maintenance  battalion,  courses  taken,  and 
results  of  qualification  tests.  However,  it  does  not  appear 
that  the  MPS  would  contain  information  about  effective  and 
ineffective  performance  such  as  time  to  diagnose  malfunctions 
and  success  or  failure  to  diagnose  malfunctions  of  various  types. 

III.  SMART:  Simulation  Model  for  Allocation  of  Resources  for  Training 
(Air  Force) 

This  database  serves  a  similar  purpose  to  that  of  the 
Army's  AIMS,  although  it  contains  only  course-level  as 
opposed  to  individual-level  data.  Course  description  data 
include  prerequisites,  training  location,  course  length,  and 
class  size,  among  other  data.  Resource  requirements  data 
consist  of  course-level  requirements  for  facilities, 
equipment,  and  personnel. 

IV.  BLTMS:  Branch  Level  Training  Management  System  (Air  Force) 

Under  development,  this  system  is  to  be  an  on-line 
system  maintained  at  the  branch  level  with  the  exact 
content  of  the  data  subject  to  the  local  training  manager's 
needs.  Among  the  data  to  be  included  are  information  on 
instructors  such  as  personnel  information,  teaching  status 
qualifications,  and  teaching  schedule.  Also  to  be  included 
are  information  pertaining  to  graduates,  and  information  on 
students,  including  personal  data,  student  status,  test 
scores,  and  other. 

V.  ORDB :  Occupational  Research  Data  Bank  (Air  Force) 

This  is  an  occupational  information  system  that  provides 
information  on  a  wide  variety  of  specialties  and  the  people 
who  perform  in  them.  Included  in  the  data  bank  is  a  history 
of  job  codes  and  their  training  patterns,  including  courses 
required  and  elective,  OJT  requirements,  and  progression  of 
training . 
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The  list  is  not  exhaustive,  but  exemplifies  the  kinds  of  databases 
already  in  existence  (or  soon  to  be).  Not  all  skills  are  covered  and 
the  type  and  detail  of  data  vary  across  services.  However,  the  above 
databases  and  others  like  them  can  provide  a  good  starting  point  for 
developing  the  necessary  data  protocols  and  administrative  coordination 
between  TPDC  and  the  services. 

Not  all  of  the  information  in  databases  such  as  those  listed  above 
need  be  included  in  the  TPDC  database  system.  A  routine  extraction  of  a 
selected  subset  of  variables  could  be  devised  and  executed  as  each  new 
wave  of  data  arrives;  the  output  could  be  transmitted  to  TPDC  on  a 
regular  basis.  The  extraction  program  could  contain  any  recodes  or 
reformats  necessary  to  make  the  data  comparable  across  services,  or  such 
revisions  could  be  done  by  TPDC  once  the  data  has  arrived  in-house.  As 
stated  above,  ideally,  if  the  individual  service  databases  collected  and 
kept  data  in  the  same  format  and  codes,  the  combining  of  training  data 
across  services  would  be  much  easier  for  TPDC.  However,  if  commonality 
is  difficult  to  achieve,  then  protocols  will  have  to  be  developed  with 
each  service  to  transform  the  data  either  before  transmission  to  TPDC  or 
upon  arrival  at  TPDC. 

REQUIREMENT  FOR  LONGITUDINAL  DATA 

Acquisition  of  longitudinal  data  is  a  key  requisite  for  an 
effective  training  database.5  Longitudinal  data  enable,  for  example, 
comparisons  of  cohorts,  evaluation  of  policy  changes,  studies  of  changes 
in  skill  requirements  over  time,  and  comparison  of  different  methods 
used  over  time  in  the  same  course.  Information  collected  by  the 
services  may  not  be  kept  over  a  long  period  of  time.  Old  data  may 
simply  be  replaced  as  updated  versions  come  in.  Thus,  the  frequency  of 
transmission  of  data  between  the  services  and  TPDC  is  a  crucial 
consideration.  Frequency  of  data  transmission  will  depend  on  the 
particular  database  from  which  the  extraction  is  being  made,  on  the 
length  of  time  that  data  are  kept,  and  on  the  work 

JThe  date  and  time  period  indicators  discussed  in  Sec.  II  enable 
the  linking  of  longitudinal  data  across  files. 
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requirements/schedules  of  the  service  personnel  handling  the  data 
extraction  and  transmission.  Data  which  change  only  annually  need  only 
be  sent  once  a  year,  whereas  data  which  come  in  every  month  might  be 
transmitted  quarterly  or  even  more  frequently  if  the  system  cannot  store 
data  for  that  long. 

CREATION  OF  AN  INTELLIGENT  DATABASE 

The  concept  of  "intelligent"  databases  is  being  advocated  by 
information  processing  experts  as  a  way  to  assist  the  assemblage  and  use 
of  large-scale  databases.  Whereas  an  ordinary  database  typically 
consists  of  computer  data  files  and  printed  codebooks,  an  intelligent 
database  has  an  automated  codebook  including  variable  definitions  and 
commentaries  about  data  sources  and  quality;  the  data  are  automatically 
edited  according  to  programmed  instructions  and  criteria  which 
themselves  are  documented;  and  summary  statistics  are  readily  available 
to  browsers,  either  in  tabular  or  graphic  form. 

An  intelligent  database  management  system  might  have  the  linkages 
among  files  already  set  up  so  that  the  user  could  simply  select  data 
elements  from  the  various  files;  the  information  across  files  would  be 
combined  automatically.  The  database  management  system  might  also 
perform  aggregations  from,  say,  individual-level  to  course-level  or  unit- 
level  or  MOS- level,  course-level  to  school-level  or  MOS-level  or 
instructor- level ,  and  so  on,  for  sets  of  variables  for  which  such 
aggregations  might  commonly  be  done. 

The  enormity  of  the  training  database  discussed  in  this  Note  almost 
certainly  requires  tape  data  storage.  Having  the  database  itself  keep 
track  of  what  is  on  which  tape  would  be  a  valuable  feature.  The  user 
would  only  need  to  specify,  say,  the  time  periods  needed  from  a  given 
file  and  the  system  would  locate  the  data.  The  same  would  be  true  for 
drawing  data  across  files--the  system  would  keep  track  of  where  the  data 
are  and  call  them  up  for  the  user. 

The  database  system  should  also  be  able  to  write  out  the  data 
requested  in  specified  formats  to  data  files  that  can  be  used  for 
statistical  analyses  or  reports.  Combined  with  the  automated  linkage 
and  aggregation  and  data  location,  this  feature  would  ease  the  burden  of 
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the  TPDC  programming  staff  when  responding  to  simple  or  standard  data 
requests  by  research  analysts.  More  complicated  requests  would  have  to 
be  handled  by  uniquely  written  data  extractions,  but  might  have  quicker 
response  since  simple  extractions  can  be  done  through  automation  of  the 
system. 


V.  CONCLUSION 


Our  proposed  architecture  for  a  TPDC  database  on  initial  skill 
training  has  been  guided  by  a  single,  overarching  principle  of  selection 
that  determines  what,  from  all  the  possible  collectible  data,  should 
constitute  the  proper  content  and  design  of  a  useful  and  flexible 
databank.  That  principle:  Training  should  be  considered  an  investment 
in  human  capital  meant  to  sustain  or  increase  military  capability.  In 
essence,  data  should  be  collected  on  aspects  of  personnel  training  that 
relate  to  or  affect  military  capability.  Moreover,  data  should  be 
collected  on  a  selected  subset  of  specialties,  as  well  as  on  both  active 
duty  and  reserve  forces.  A  TPDC  database  should  also  permit  maximum 
linkage  among  its  files  through  the  use  of  several  identifiers. 

The  TPDC  database  provides  a  foundation  for  the  evaluation  of  1ST 
effectiveness.  Researchers  can  capitalize  on  natural  experiments  or 
utilize  the  database  to  provide  an  extensive  quantitative  background  for 
controlled  experiments.  The  complex  and  varied  nature  of  the  database 
also  allows  for  applications  beyond  the  evaluation  of  training 
effectiveness.  Some  applications  are  directly  related,  some  indirectly 
related  to  training  issues.  To  illustrate,  we  discussed  five 
applications  for  the  relevant  files  in  the  1ST  database:  (1) 
determining  training  loads  for  budgeting,  (2)  matching  weapons  system 
acquisition  and  training  requirements,  (3)  managing  skill  shortages,  (4) 
selecting  personnel  into  military  occupations,  and  (5)  determining  the 
value  of  military  training  to  the  individual. 

An  effective  TPDC  database--that  is,  one  allowing  for  varied 
applications  over  time--roust  take  into  account  certain  structural 
aspects  in  its  long-term  development:  the  necessity  for  interaction 
among  the  different  services'  databanks,  the  requirement  that  the  data 
be  longitudinal,  and  the  necessity  for  an  intelligent  database.  In 
short,  the  TPDC  database  should  contain  a  series  of  files  that  can  be 
connected  or  linked  in  different  combinations  to  serve  different  kinds 
of  research  and  analysis. 
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Compiling  the  kind  of  database  we  have  outlined  and  discussed  is  a 
vast  effort  that  is  likely  to  proceed  incrementally  at  best  over  many 
years.  We  are  not  certain  of  the  best  place  to  begin,  but  it  seems  that 
the  most  readily  accessible  data  pertain  to  the  course  and  the 
individual.  In  contrast,  it  may  take  many  years  to  develop  measures  of 
job  performance  and  unit  productivity,  so  the  TPDC  database  is  likely  to 
evolve  slowly  in  those  directions.  Still,  it  is  important  to  begin  with 
a  unified  perspective  to  ensure  that  incremental  efforts  result  in  an 
integral  whole.  As  emphasized,  selective  data  collection  by  occupation 
may  prove  to  be  the  most  cost  effective  way  to  proceed.  Between  the 
various  areas  of  application  mentioned  above,  it  is  risky  to  assign 
priorities  for  data  collection.  Today,  with  high  personnel  quality  and 
retention,  it  may  seem  least  important  to  assemble  data  on  the  value  of 
military  training  to  the  individual.  But  over  the  long  term,  retention 
could  become  a  problem,  and  given  the  long  lead  times  in  data  base 
development,  it  seems  prudent  to  pursue  all  five  areas  of  application 
simultaneously.  This  suggestion  becomes  more  practical  the  narrower  the 
band  of  occupations  selected  for  extensive  data  collection.  We  stress 
that  the  successful  creation  of  an  efficient,  flexible,  useful  database 
will  require  the  cooperation  of  all  the  services,  as  well  as  of  OSD. 
Their  cooperation  will  depend  on  two  conditions:  (1)  evidence  that 
useful  studies  can  be  done  from  the  kind  of  data  in  an  1ST  database,  and 
(2)  assurances  that  the  content  and  consequences  of  these  studies  will 
not  threaten  the  services'  mission  to  increase  their  capability.  The 
services  may  find  some  reassurance  in  the  example  of  the  Defense 
Manpower  Data  Center,  created  10  years  earlier;  its  data  files, 
assembled  with  the  help  of  the  services,  have  provided  a  detailed  record 
of  personnel  trends  and  the  effects  of  policy  and  economic  changes  on 
those  trends,  and  both  the  OSD  and  the  services  have  used  the  files  for 
a  large  number  of  valuable  studies. 

Another  problem  in  creating  an  1ST  database  is  the  lack  of  general 
agreement  on  how  to  measure  performance.  The  cooperation  of  the 
services-- for  their  individual  and  collective  benefits--will  also  be 
needed  to  overcome  such  difficulties.  Despite  such  problems,  the 


architecture  we  propose  for  the  TPDC  databank  would  accommodate  many 
kinds  of  information  from,  and  useful  to,  all  the  services,  including 
data  on  course  training,  the  selection  of  individuals  into  military 
occupations,  recruit  performance  in  1ST,  on-the-job  performance,  unit 
productivity,  and  the  relations  among  these  areas.  The  services  should 
find  many  fruitful  uses  for  the  database  as  it  grows  and  evolves. 
Furthermore,  it  should  support  experimentation  in  different  kinds  of 
research  and  analysis  and  yield  heretofore  unexpected  findings  of 
benefit  to  military  managers  and  planners. 
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